Introduction to Auditing AI Systems

Course details

AI regulation is here, so you may be wondering how to adapt. And if you’re an enterprise organization, startup, or AI practitioner, you probably already know that there are very few existing training resources for technical teams. In this course, instructor Ayodele Odubela gives you a hands-on overview of how to assess AI for bias and discrimination to create fairer AI systems.

Explore the fundamentals of the latest regulations governing the use of AI technology, as well as how to navigate the different phases of AI audits in technical terms. Learn how federal discrimination laws can impact AI systems, how to audit both high- and low-risk AI, and how to collect, develop, or purchase benchmark data for auditing and policy review. Ayodele shows you the basics of calculating model fairness and what principles to prioritize and why, including explainability, transparency, compliance, and documentation. Upon completing this course, you’ll be more aware of how to use generative AI tools to mitigate algorithmic bias.

Concepts

0. Introduction

01 - Welcome to the new world of AI audits

1. New Paradigm of AI Audits

02 - What is an AI audit
03 - How are audits used
04 - The state of AI legislation
05 - Ethics of scoring and classifying humans

2. Why Audit AI Systems

06 - AI audit limitations and opportunities
07 - Development workflows
08 - AI performance
09 - Statistical parity

3. Data for AI Audits

10 - Data for auditing AI
11 - Sources of bias in data
12 - Types of bias and data sampling methods

4. Principles for AI Audits

13 - Why explainability matters
14 - Levels of transparency
15 - Responsible AI principles - Compliance
16 - Preparing for AI regulation

5. Model Audits

17 - Types of model audits
18 - Stages of a model audit
19 - Model audit - Home loans
20 - Auditing training data
21 - Audit outcomes - Explainability statements
22 - Continuous audits

Conclusion

23 - Generative AI
24 - Next steps