Applied AI Auditing in Python

Course details

AI regulation has arrived, and while there are great high-level strides towards operationalizing AI principles, practitioners must adopt the practice of auditing AI systems to meet compliance and transparency standards. This course guides data scientists and machine learning engineers through the technical process of auditing an AI system, step-by-step. This course is a hands-on, technical course that shows you how to quantify unfairness and disparities to uncover bias and develop fairer AI systems. Instructor Ayodele Odubela explains how to plan, execute, and report on AI audits. Learn the difference between data and algorithm auditing, frameworks for scalable audits, and how context and historical bias play a role in making technical recommendations.

Learning objectives
Learn how to audit high- and low-risk AI systems
Collect, design, and mange benchmark data for auditing
Calculate and perform a disparity analysis
Determine model fairness with the Fairlearn Python package
Decide which fairness metric to use
Make technical recommendations post-audit

Concepts

Introduction

Get responsible with AI - Auditing AI systems in Python
What you should know
Using the exercise files and datasets

Introduction to Applied AI Auditing

AI auditing for compliance and fairness
AI audit stakeholders
Localized fairness and compliance

Data Auditing

How to collect benchmark datasets
Ethical and inclusive data collection
Explore a dataset for representation
Data auditing example
Challenge - Audit a dataset
Solution - Methods for increasing representation in data

AI Model Auditing

Tools for AI audits
Scoping an AI audit
Model audit setup
Audit your classifier for fairness
Challenge - Audit a classifier
Solution - Audit a classifier

System Audits and Error Analysis

Red teaming
Error analysis
Challenge - Error analysis
Solution - Error analysis

Audit Artifacts

Making audit recommendations
Sharing audit results and increasing accountability
Algorithmic recourse
Algorithmic design history file

Conclusion

Thanks for watching