Special offers now — see discounted courses.
day
:
hour
:
min
:
sec
See special offers
Hands-On Introduction: Data Engineering

Hands-On Introduction: Data Engineering

1h 29mBeginner2023-04-28

Authors

Vinoo Ganesh

Vinoo Ganesh

CEO and founder of Stealth Startup

Course details

Today’s world is flooded with data, which puts businesses up to task. In response, organizations, companies, and employees around the world have adopted data-driven decision-making techniques. However, the vast majority of new data practitioners haven’t been formally trained. They don’t know how to build and construct stable data pipelines that can function effectively at speed and scale. Moreover, they haven't been given the framework(s) to distill data-oriented tasks into discrete components.

In this course, instructor Vinoo Ganesh gives you an overview of the fundamental skills you need to know to solve complex data problems in a scalable, productive way. Explore the core principles of the data engineer toolkit—including ELT, OLTP/OLAP, orchestration, DAGs, and more—as well as how to set up a local Apache Airflow deployment and full-scale data engineering ETL pipeline. Along the way, Vinoo helps you boost your technical skill set using real-world, hands-on scenarios.

Skills covered

Data EngineeringData ScienceOne-Off

Concepts

0. Introduction

  • 01 - Hands-on data engineering
  • 02 - Background knowledge
  • 03 - The history of data in the enterprise
  • 04 - How to use GitHub Codespaces with this course

1. Getting Your Bearings

  • 05 - Data engineering and data pipelines
  • 06 - Orchestration - In the modern sense
  • 07 - Extract, transform, load (ETL)
  • 08 - Tasks, DAGs, and dependencies

2. Airflow Introduction

  • 09 - Introduction to Airflow
  • 10 - Installing Airflow
  • 11 - Running the Airflow webserver and scheduler
  • 12 - Adjusting Airflow configuration settings
  • 13 - Build a 1 Task DAG
  • 14 - Build a 2 Task DAG

3. Basic ETL in Airflow

  • 15 - ETL in Airflow
  • 16 - A real-world data engineering task
  • 17 - Extracting data manually
  • 18 - Extracting data with Airflow
  • 19 - Transforming data manually
  • 20 - Transforming data with Airflow
  • 21 - Loading data manually
  • 22 - Loading data with Airflow
  • 23 - Building an ETL DAG with Airflow
  • 24 - Challenge - Review ETL questions
  • 25 - Solution - Solutions to ETL questions

Conclusion

  • 26 - The future of data engineering

Related courses

Related learn paths

About us

LyndaKade is a leading learning platform that helps people learn business, software, technology, and creative skills to achieve personal and professional goals.

Phone numberAparat ChannelTelegram SupportTelegram ChannelInstagram Page

All rights to this site belong to LyndaKade.

Terms of Service|Privacy Policy

نماد الکترونیک enamad در صورت اتصال با آی‌پی داخل کشور، نمایش داده خواهد شد.
logo-samandehi - لوگو ساماندهی
zarinpal
zibal