Databricks Certified Data Engineer Associate Cert Prep: 4 Production Pipelines
19mIntermediate2024-01-24
Authors
Noah Gift
MLOps Expert | Solopreneur | Author | Adjunct Professor | CTO
Course details
The Databricks Certified Data Engineer Associate certification exam assesses your ability to use the Databricks Lakehouse Platform to complete basic data engineering tasks. It also assesses your ability to use Apache Spark SQL and Python to perform multihop architecture ETL tasks that incrementally process new data from a variety of data sources to power analytic applications and dashboards in the Lakehouse.
In this course, instructor Noah Gift provides hands-on, practical instruction on the core concepts and skills covered in the fourth section of the exam: Production Pipelines. Explore the fundamentals of creating and managing data pipelines with Delta Live Tables, including vacuuming and garbage collection, table documentation, automated pipelines, continuous versus triggered pipelines, configuring Auto Loader, querying pipeline events, and much more.
In this course, instructor Noah Gift provides hands-on, practical instruction on the core concepts and skills covered in the fourth section of the exam: Production Pipelines. Explore the fundamentals of creating and managing data pipelines with Delta Live Tables, including vacuuming and garbage collection, table documentation, automated pipelines, continuous versus triggered pipelines, configuring Auto Loader, querying pipeline events, and much more.
Skills covered
DatabricksData EngineeringData AnalysisCert PrepData ScienceBusiness Analysis and StrategyBusiness Software and Tools
Concepts
1. Data Pipelines with Delta Live Tables
- 01 - Vacuuming and garbage collection
- 02 - Table documentation
- 03 - Automated pipelines with Delta Live Tables
- 04 - Delta Live Tables components
- 05 - Continuous vs. triggered pipelines
- 06 - Configuring Auto Loader
- 07 - Querying pipeline events
- 08 - End-to-end example of Delta Live Table