Introduction to Modern Data Engineering with Snowflake

Course details

In this technical, hands-on course, learn how to build and orchestrate modern data pipelines with Snowflake. The course focuses on practical concepts and tools to get you up and running quickly. Learn how to build and maintain data pipelines in Snowflake, including data ingestion, transformation, delivery, and orchestration. Throughout the course, follow along with the instructor using a combination of Snowflake, Visual Studio Code, GitHub, the command line, and more. The course is supplemented with readings containing plenty of resources to level up your understanding of specific concepts.

Learning objectives
Ingest data into Snowflake at scale using a variety of techniques.
Perform data transformations with SQL or Python.
Extend data transformations with user-defined functions, stored procedures, streams, and Snowflake Dynamic Tables.
Deliver valuable data products through Snowflake Marketplace, Streamlit in Snowflake, and Snowflake Native Applications.
Orchestrate pipelines using Tasks and DAGs.

Concepts

Introduction

The explosion of data and the demand for insights
What we ll cover in this course

Modern Data Engineering with Snowflake

Modern data engineering with snowflake
You ve probably done some data engineering in the past
Preparing your development environment
Build a really simple data pipeline in snowflake

Batch Data Ingestion with Snowflake

What is data ingestion
Batch ingestion with snowflake
Loading data from snowflake marketplace
Loading data using snowflake s web interface
Optimize compute resources for efficient batch ingestion
Loading data using snowflake CLI
Loading data using the COPY INTO command
Scenario briefing and account setup
Ingesting data from other data systems using connectors
Recap and best practices for batch ingestion

Data Transformations with Snowflake

What are data transformations
Data transformations with SQL
Data transformations with snowpark
Computations with user-defined functions
Efficient transformations with streams
Complex procedural logic with stored procedures
Automatic transformations with dynamic tables
Data transformations in visual studio code (optional)
Recap and best practices for data transformations

Delivering data products with Snowflake

Delivery of data products
Data sharing on snowflake marketplace
Streamlit in snowflake applications
Snowflake native applications
Recap and best practices for data product delivery

Orchestrating Continuous Data Pipelines with Snowflake

What is orchestration
Automation with tasks
Orchestration with DAGs
Recap and best practices for pipeline orchestration
Conclusion