Snowpark for Data Engineers
1h 51mIntermediate2024-09-10
Authors
Janani Ravi
Certified Google Cloud Architect and Data Engineer
Course details
Take a deep dive into Snowpark Python in Snowflake, the library set specifically designed for data engineers and professionals seeking to leverage the capabilities of the Snowflake managed data platform. Join instructor Janani Ravi as she shows how to get up and running with Snowpark, from setting up a Snowflake trial account to writing Snowpark handlers, performing data transformations, and working with both structured and semistructured data. Along the way, explore more advanced concepts such as creating and managing user-defined functions (UDFs), user-defined table functions (UDTFs), and stored procedures, as well as how to install necessary packages, access custom packages, and connect to Snowflake from a Jupyter notebook. By the end of this course, you’ll be prepared to start manipulating data frames, performing data engineering tasks, and implementing complex data processing functions within Snowflake.
Skills covered
SnowflakeCloud StorageData EngineeringDatabase DevelopmentDatabase ManagementCloud ServicesCloud ComputingData ScienceSoftware DevelopmentOne-Off
Concepts
0. Introduction
- 01 - Snowpark for data engineers
- 02 - Prerequisites
1. Snowpark Handlers in Python Worksheets
- 03 - Getting set up with Snowflake
- 04 - Writing our first Snowpark handler
- 05 - Returning a table from the Snowpark handler
- 06 - Transformation with Snowpark data frames
- 07 - Reading files from a stage
- 08 - Defining multiple functions in a Python worksheet
- 09 - Installing Anaconda packages in a session
- 10 - Accessing custom packages from a stage
2. Using Snowpark DataFrames
- 11 - Using Snowpark from a locally running Jupyter Notebook
- 12 - Data transformations using Snowpark DataFrames
- 13 - Performing union operations on DataFrames
- 14 - Performing joins
- 15 - Creating views
- 16 - Working with semi-structured data
3. Working with UDFs
- 17 - UDFS, UDTFs, and Stored procedures
- 18 - Creating anonymous UDFs in Snowpark
- 19 - Creating named UDFs in Snowpark
- 20 - Accessing external packages in AUDF
- 21 - Understanding temporary UDFs
- 22 - Creating and invoking permanent UDFs
4. Working with UDTFs
- 23 - Creating and registering a UDTF
- 24 - Invoking a UDTF in Python and SQL
- 25 - Creating a table and uploading data
- 26 - Implementing a full-fledged UDTF
- 27 - Invoking UDTFs with different arguments
5. Working with Stored Procedures
- 28 - Creating and registering stored procedures
- 29 - Creating parameterized stored procedures
- 30 - Stored procedures with loops and conditional logic
- 31 - Deploying a stored procedure
Conclusion
- 32 - Summary and next steps