MLOps with Databricks

Course details

In this course, MLOps expert Maria Vechtomova introduces the components and principles that you must understand to successfully deploy machine learning models to production on Databricks. Dive into the step-by-step process of using Feature Engineering in Unity Catalog, tracking model experiments in mlflow, registering a model in Unity Catalog, and deploying your model using Databricks model serving. Explore the use cases where Feature Serving can be used and find out how to deploy a Feature Serving endpoint. Plus, learn how to package your code, deploy your project using Databricks Asset Bundles, and monitor your ML application using inference tables and Lakehouse monitoring.

Learning objectives
Explain main components and principles required to deploy machine learning models to production on Databricks.
Identify how to use experiment tracking system, model registry, feature engineering, model/feature serving, and other features required to deploy ML applications.
Articulate how to package your Python code using best practices and deploy your project using Databricks Asset Bundles.
Review how to monitor your ML applications.

Concepts

Introduction

Reduce time to production with MLOps

MLOps Components and Principles

MLOps components
MLOps principles
MLOps components on Databricks

MLflow

Introduction to MLflow
Set up the environment and prepare the dataset
MLflow experiment tracking
Model logging in MLflow
Custom models in MLflow
Register models in Unity Catalog

Feature Engineering on Databricks

Introduction to feature engineering
Feature table and feature function
Train, log, and register models that use on-demand features

Feature and Model Serving

When to use Feature Serving vs. Model Serving
Create a FeatureSpec for Feature Serving
Deploy serving endpoints
Send requests to the serving endpoints

Deploy Application Using Databricks Asset Bundles

Getting started with DABs
Define Databricks workflow components
Define and deploy the Databricks Asset Bundle

Monitor Your Endpoints Using Inference Tables and Lakehouse Monitoring

Introduction to Lakehouse Monitoring
Set up, analyze, and process the inference table
Set up Databricks Lakehouse Monitoring

Conclusion

Mastering MLOps - Your journey begins