Introduction to Modern Data Engineering with Snowflake

Course details

In this technical, hands-on course, learn how to build and orchestrate modern data pipelines with Snowflake. The course focuses on practical concepts and tools to get you up and running quickly. Learn how to build and maintain data pipelines in Snowflake, including data ingestion, transformation, delivery, and orchestration. Throughout the course, follow along with the instructor using a combination of Snowflake, Visual Studio Code, GitHub, the command line, and more. The course is supplemented with readings containing plenty of resources to level up your understanding of specific concepts.

Learning objectives
Ingest data into Snowflake at scale using a variety of techniques.
Perform data transformations with SQL or Python.
Extend data transformations with user-defined functions, stored procedures, streams, and Snowflake Dynamic Tables.
Deliver valuable data products through Snowflake Marketplace, Streamlit in Snowflake, and Snowflake Native Applications.
Orchestrate pipelines using Tasks and DAGs.

Concepts

0. Introduction

01 - The explosion of data and the demand for insights
02 - What we ll cover in this course

1. Modern Data Engineering with Snowflake

03 - Modern data engineering with snowflake
04 - You ve probably done some data engineering in the past
05 - Preparing your development environment
06 - Build a really simple data pipeline in snowflake

2. Batch Data Ingestion with Snowflake

07 - What is data ingestion
08 - Batch ingestion with snowflake
09 - Loading data from snowflake marketplace
10 - Loading data using snowflake s web interface
11 - Optimize compute resources for efficient batch ingestion
12 - Loading data using snowflake CLI
13 - Loading data using the COPY INTO command
14 - Scenario briefing and account setup
15 - Ingesting data from other data systems using connectors
16 - Recap and best practices for batch ingestion

3. Data Transformations with Snowflake

17 - What are data transformations
18 - Data transformations with SQL
19 - Data transformations with snowpark
20 - Computations with user-defined functions
21 - Efficient transformations with streams
22 - Complex procedural logic with stored procedures
23 - Automatic transformations with dynamic tables
24 - Data transformations in visual studio code (optional)
25 - Recap and best practices for data transformations

4. Delivering data products with Snowflake

26 - Delivery of data products
27 - Data sharing on snowflake marketplace
28 - Streamlit in snowflake applications
29 - Snowflake native applications
30 - Recap and best practices for data product delivery

5. Orchestrating Continuous Data Pipelines with Snowflake

31 - What is orchestration
32 - Automation with tasks
33 - Orchestration with DAGs
34 - Recap and best practices for pipeline orchestration
35 - Conclusion