MLOps Essentials: Monitoring Model Drift and Bias
1h 5mIntermediate2023-10-06
Authors
Kumaran Ponnambalam
Working with data for 20+ years
Course details
As more and more ML models are developed and deployed, the need arises to ensure that the models are effective and safe and that they perform as desired. Model monitoring, a core function of MLOps, helps data scientists and MLOps engineers to meet this need. In this course, data analytics expert Kumaran Ponnambalam discusses the types of monitoring needed for ML models. He deep dives into model drift monitoring and bias. For model drift, Kumaran goes over the types of drift monitoring and their causes. He explains different techniques for drift monitoring and how to execute them in python using open source libraries. For bias, Kumaran highlights various sources of bias and their impact. He also analyzes bias in python with open source libraries. Finally, he recommends some best practices for drift and bias monitoring.
Skills covered
KerasTensorFlowMachine LearningGoogleFoundationsArtificial Intelligence (AI)Open Source
Concepts
Introduction
- The need for model monitoring
- Setting up the exercise files
Introduction to Model Monitoring
- ML models in production
- Challenges with serving models in production
- Metrics to monitor
- Data for model monitoring
Model Drift Basics
- Introduction to model drift
- Concept drift
- Feature drift
- What causes drift
- Drift remediation process
Detecting Model Drift
- Detecting concept drift
- Concept drift detection example
- Detecting feature drift
- Feature drift detection example
- Detecting drift in text and images
- Software for drift detection
Drift Monitoring Process and Best Practices
- Drift monitoring pipeline
- Analyzing drift trends
- Discovering root causes for drift
- Retraining to overcome drift
Introduction to Model Bias
- Fairness and bias
- Fairness in ML
- Sources of ML bias
- Protected attributes
- Demographic parity
Bias Detection and Best Practices
- Bias detection techniques
- Equal opportunity score
- EOS example
- Bias detection software
- Overcoming bias in ML
Conclusion
- Next steps