Data Quality Testing with Great Expectations

Course details

Data quality is the foundation of reliable analytics and machine learning—even the most advanced analytics platform or AI model is worthless without accurate, trustworthy data. In this course, expert data engineer Sam Bail teaches you how to use Great Expectations, a powerful open-source framework for testing and validating data. Explore when and where data quality testing matters most, and find out how to configure both the open-source and cloud-based versions of Great Expectations for your workflows. Leverage hands-on examples to implement data quality tests, interpret the results, and debug common issues. This course equips you to build robust, trustworthy data pipelines that catch data problems before they cause downstream damage.

Learning objectives
Understand when and where data quality testing is important.
Install and configure Great Expectations (both the open source and cloud-based versions) and implement data quality tests.
Know how to interpret and debug failed data quality tests.

Concepts

Introduction

Data quality testing with GX introduction

Introduction to Data Quality Testing

Why data quality matters
What do we mean by data quality
From manual to automated testing
What is Great Expectations (GX)

Getting Started with GX Core

Creating a GX data context
Connecting to data
Creating a data source
Creating a data asset
Creating a batch definition
Defining Expectations
Running Validations
Using Data Docs

Debugging and Maintaining Data Quality

Triggering actions with checkpoints
Understanding data test failures
Root cause analysis of test failures
Approaches to debugging and fixing data quality issues
Creating fuzzy expectations
Updating and deleting Expectations in an Expectation Suite
Creating custom Expectations
Monitoring ongoing data quality

Integrating GX in Production Data Pipelines

How GX fits into your data stack
GX workflows
GX Cloud overview

Conclusion

Next steps