Threat Modeling for AI/ML Systems
57mAdvanced2024-04-25
Authors
Adam Shostack
Consultant, Entrepreneur, Technologist, and Game Designer
Course details
So much is happening in the world of AI right now that it can be hard to make sense of what’s what. And if you’re a developer, product manager, program manager, or site reliability engineer, you’re expected to deliver secure systems in a practical way. This course is designed to give technologists a durable framework for thinking about what can go wrong with an AI system and how to respond to deliver actionable results. Explore some of the best available frameworks for understanding, categorizing, and discovering security attacks broadly. Instructor Adam Shostack provides an overview of threat modeling, how it fits into the ML and AI systems, and how to create and maintain secure, trustworthy systems.
Skills covered
Software Development SecurityMachine LearningArtificial Intelligence FoundationsArtificial Intelligence (AI)CybersecurityOne-Off
Concepts
Introduction
- Threat modeling introduction
- What you should know
Threat Modeling Overview
- Threat modeling is important when building AI systems
- The four-question framework structures your work
- Anyone can threat model and you should, now
- Trustworthy AI - Threat modeling is better than principles
What Are You Working on with ML
- ML for business, offense, defense, and software
- Draw your architecture
- Deployment architectures influence your threats
- Training data is a crucial variable
- The stochastic parrot
What Can Go Wrong with ML Security
- The OWASP Top Ten as a checklist
- The Berryville Institute Exhaustive List
- Microsoft's frameworks for security flaws
- Prompt injection
- Embarrassing and hostile results
What Can Go Wrong with AI - Trustworthiness
- NIST Framework
- EU's AI Act
- Current harms
- Scenarios
What Are You Going to Do about It
- Specific frameworks
- Mitigations advance faster than threats
- Deploying new technology isn't a one-and-done
Conclusion
- Next steps