Modern Data Engineering Essentials by Pearson

Course details

Data is everywhere, and it keeps getting bigger, so you need to ensure your skills are fully up-to-date. In this course, discover the essentials, core concepts, and key issues surrounding big data and AI systems. From data lakes to database engineering, data transformation, and orchestration, this course covers everything you need to know to get up and running in this growing field. Along the way, gain hands-on experience using cutting-edge data tools and workflows. By the end of this course, you’ll also be apprised of the education, experience, and credentials required to continue to develop your skills in the modern data world. This course is an ideal fit for developers, data scientists, and engineers who are interested in the data side of big data systems.

Learning objectives
Apply data engineering concepts.
Load data into DuckDB.
Set up and use dbt.
Orchestrate data engineering workflows using Airflow, Dagster, and Prefect.
Put it all together integrating Dagster and dbt.
Maximize your education, experience, and skills to succeed in the modern data engineering world.

Concepts

Introduction

Modern data engineering

Modern Data Lakes

A brief history of big data
The modern data landscape
Data warehouses and lakehouses in the context of AI

Fundamentals of Database Engineering

Relational versus nonrelational data models
Row versus columnar data stores
Hands-on demo - Load data into DuckDB
Design for performance

Data Transformation

Extract, transform, load versus extract, load, and transform
SQL or Python
Hands-on demo - dbt core setup

Workflow Orchestration

Putting it all together - Dagster and dbt
Hands-on demo - Dagster assets

Conclusion - The Future of Data Engineering

Degree or decree
Range - Why generalists will triumph over specialists
Design for performance