Databricks Certified Data Engineer Associate Cert Prep: 3 Incremental Data Processing

Databricks Certified Data Engineer Associate Cert Prep: 3 Incremental Data Processing

28mIntermediate2024-01-24

Authors

Noah Gift

Noah Gift

MLOps Expert | Solopreneur | Author | Adjunct Professor | CTO

Course details

The Databricks Certified Data Engineer Associate certification exam assesses your ability to use the Databricks Lakehouse Platform to complete basic data engineering tasks. It also assesses your ability to use Apache Spark SQL and Python to perform multihop architecture ETL tasks that incrementally process new data from a variety of data sources to power analytic applications and dashboards in the Lakehouse.

In this course, instructor Noah Gift provides hands-on, practical instruction on the core concepts and skills covered in the third section of the exam: Incremental Data Processing. Explore the fundamentals of data management with Delta Lake, including efficient data transformation with Spark SQL, using Catalog Explorer, creating tables from files, querying external data sources and inspecting tables, ACID transactions, Z-ordering, and more.

Skills covered

DatabricksData EngineeringData AnalysisCert PrepData ScienceBusiness Analysis and StrategyBusiness Software and Tools

Concepts

Data Management with Delta Lake

  • Efficient data transformation with Spark SQL
  • Using Catalog Explorer
  • Creating tables from files
  • Querying external data sources
  • Inspecting tables
  • Reliable data pipelines with Delta Lake
  • ACID transactions
  • Z-ordering optimization