Databricks Certified Data Engineer Associate Cert Prep

Course details

Databricks unifies data engineering, analytics, and machine learning on a single platform built around Apache Spark. This course prepares you for the Databricks Certified Data Engineer Associate exam. Explore how Databricks fits into modern data engineering and integrates with Apache Spark. Work through Spark architecture and execution, data transformations, actions, and lazy evaluation, then move into the performance optimization techniques that data-heavy applications require. Learn Delta Lake for data management, including ACID transactions and time travel, alongside core warehousing concepts such as OLAP and OLTP. Explore advanced features like Unity Catalog for governance and Delta Live Tables for orchestration. Through practical demonstrations, create and manage notebooks, ingest and transform data, and optimize performance using partitions and parallelism. At the end of this course, a capstone project helps you tie the core concepts together and gain the experience you need to pass the exam.

Learning objectives
Gain proficiency in Databricks UI and workflow management.
Master Spark fundamentals and understand execution plans.
Learn how to optimize data processing with partitions and parallelism.
Develop skills in managing data with Delta Lake and implementing ACID transactions.
Create and orchestrate data pipelines using Delta Live Tables and GitHub integration.
Successfully complete the capstone project to showcase your skills.

Concepts

Introduction

Course introduction

Getting Started with Databricks

Section introduction
What is Databricks and why it exists
Demonstration - Databricks editions and free signup UI walkthrough
Demonstration - Exploring the Databricks dashboard
Demonstration - Creating your first notebook and ingesting data
Summary

Understanding Spark Fundamentals

Section introduction
What is Apache Spark and PySpark
Spark architecture - Driver, executors, and cluster concept
Spark session - The entry point of Spark
Demonstration - Basic DataFrame operations
Demonstration - Switching between PySpark and SQL
Demonstration - Magic commands in Databricks and Spark session
Demonstration - Load and transform data using Apache Spark DataFrames
Summary

Understanding Spark Execution Basics

Section introduction
Understanding Spark plans with explain()
Demonstration - Understanding Spark execution plans with explain()
Transformations vs. actions and lazy evaluation
Narrow vs. wide transformations
Summary

Performance Basics

Section introduction
Partitions and parallelism - Conceptual
Repartition vs. coalesce
Demonstration - Repartition vs. coalesce
Summary

Data Warehousing Fundamentals

Section introduction
What is a data warehouse - OLTP vs. OLAP
Understanding data warehouse, data lake, and lakehouse
Why Databricks is used for analytics workloads
Basic warehouse concepts - Fact, dimension, and star schema high level
How performance, partitions, and file layout matter in warehouses
Summary

Delta Lake and Data Management

Section introduction
Project introduction - Business use case
What is Delta Lake and why it matters
Understanding ACID transactions and time travel
Managed vs. external tables
Demonstration - Dataset preparation
Demonstration - End-to-end external data integration with S3 and type of tables
Demonstration - Delta Lake ACID transactions and time travel
Medallion architecture (bronze silver gold)
Demonstration - End-to-end medallion architecture walkthrough
Summary

Governance, BI, and Pipelines

Section introduction
Introduction to Unity Catalog - Why governance matters
Demonstration - Data governance in Databricks workspace
Data understanding and simple data processing flow
Demonstration - Simple end-to-end data flow using Delta Live Tables
Demonstration - Understanding Delta Live Tables pipeline settings
Demonstration - Getting insights with Genie and BI dashboards
Summary

Orchestration in Databricks

Section introduction
Demonstration - Connecting GitHub with Databricks
Introduction to orchestration in Databricks
Demonstration - Creating and scheduling Databricks jobs
Summary

Capstone Project

Final project walkthrough
Demonstration - Capstone project, part 1
Demonstration - Capstone project, part 2
Demonstration - Capstone project, part 3

Conclusion

Course conclusion