AI Product Security: Testing, Validation, and Maintenance

Course details

This course focuses on the requirements for testing AI applications. It covers a review of the issues that can occur and the tools that are used to detect them. A formal testing management system is introduced with an example to show how it can be used to set up test cases and record the results of test runs. In addition, the course covers the use of AI red-team scanners and prompt/command injection tools that can be used to validate the existence and correct operation of guardrails in the AI model.

Learning objectives
Analyze AI system vulnerabilities using testing tools such as red-team scanners and prompt injection techniques to identify potential security risks.
Evaluate AI application guardrails through systematic testing procedures and red teaming approaches to verify their effectiveness.
Design comprehensive test cases for AI applications using KiwiTCMS to ensure systematic coverage of security concerns.
Implement operational testing programs that address responsible AI principles and security requirements throughout the AI system lifecycle.
Apply testing methodologies and tools to detect and document AI-specific problems including prompt injection, data poisoning, and model manipulation.

Concepts

0. Introduction

01 - Ensuring safe and responsible use of AI
02 - What you should know
03 - Disclaimer

1. Test Case Management

04 - Introduction to AI security
05 - Security testing for AI applications
06 - Setting up a testing lab
07 - Introduction to HuggingFace
08 - Managing local models with ollama
09 - Test case management with KiwiTCMS
10 - Security testing with KiwiTCMS
11 - Understanding AI threats
12 - Testing requirements in AI standards

2. Understanding Attacks

13 - Installing the vulnerable LLM application
14 - Testing for prompt injection
15 - Challenges with agentic code
16 - Injecting thoughts
17 - Solving the LOTR challenge
18 - Breaking the rules
19 - Predicting secrets
20 - Getting a shell
21 - Compromise during download
22 - The ART of AI testing

3. Responsible and Secure AI Model Testing

23 - AI testing tools
24 - Introduction to DeepEval
25 - Testing for relevance
26 - Testing for toxicity
27 - Vulnerability scanning with garak
28 - Scanning pickle files
29 - All along the watchtower
30 - Advanced scanning for malicious models

4. Red Teaming

31 - What is AI red teaming
32 - Preparing the target
33 - Red teaming our AI appliance

5. Operational Testing

34 - AI guardrails
35 - Running the LLM-guard
36 - Hidden layers of AI
37 - Testing the EnkryptAI guardrail

Conclusion

38 - What's next