Securing Generative AI: Strategies, Methodologies, Tools, and Best Practices

Course details

This course offers a comprehensive exploration into the crucial security measures necessary for the deployment and development of various AI implementations, including large language models (LLMs) and retrieval-augmented generation (RAG). Discover key considerations and mitigations to reduce the overall risk in organizational AI system development processes. Author and tech trainer Omar Santos covers the essentials of secure-by-design principles, focusing on security outcomes, radical transparency, and building organizational structures that prioritize security. Along the way, learn more about AI threats, LLM security, prompt injection, insecure output handling, red team AI models, and more. By the end of this course, you’ll be prepared to wield your newly honed skills to protect RAG implementations, secure vector databases, select embedding models, and leverage powerful orchestration libraries like LangChain and LlamaIndex.

Learning objectives
Explore security for deploying and developing AI applications, retrieval-augmented generation (RAG), agents, and other AI implementations.
Leverage hands-on practical skills drawn from real-world AI and machine learning cases.
Incorporate key security considerations at every stage of AI development, deployment, and operation.

Concepts

0. Introduction

01 - Securing generative AI - Introduction

1. Introduction to AI Threats and LLM Security

02 - Learning objectives
03 - Understanding the significance of LLMs in the AI landscape
04 - Exploring the resources for this course - GitHub repositories and others
05 - Introducing retrieval augmented generation (RAG)
06 - Understanding the OWASP Top 10 risks for LLMs
07 - Exploring the MITRE ATLAS (adversarial threat landscape for artificial intelligence systems) framework
08 - Understanding the NIST taxonomy and terminology of attacks and mitigations

2. Understanding Prompt Injection Insecure Output Handling

09 - Learning objectives
10 - Defining prompt injection attacks
11 - Exploring real-life prompt injection attacks
12 - Using ChatML for OpenAI API calls to indicate to the LLM the source of prompt input
13 - Enforcing privilege control on LLM access to back-end systems
14 - Best practices around API tokens for plugins, data access, and function-level permissions
15 - Understanding insecure output handling attacks
16 - Using the OWASP ASVS to protect against insecure output handling

3. Training Data Poisoning, Model Denial of Service Supply Chain Vulnerabilities

17 - Learning objectives
18 - Understanding training data poisoning attacks
19 - Exploring model denial of service attacks
20 - Understanding the risks of the AI and ML supply chain
21 - Best practices when using open-source models from Hugging face and other sources
22 - Securing Amazon Bedrock, Amazon SageMaker, Microsoft Azure AI services, and other environments

4. Sensitive Information Disclosure, Insecure Plugin Design, and Excessive Agency

23 - Learning objective
24 - Understanding sensitive information disclosure
25 - Exploiting insecure plugin design
26 - Avoiding excessive agency

5. Overreliance, Model Theft, and Red Teaming AI Models

27 - Learning objectives
28 - Understanding overreliance
29 - Exploring model theft attacks
30 - Understanding red teaming of AI models

6. Protecting Retrieval Augmented Generation (RAG) Implementations

31 - Learning objectives
32 - Understanding RAG, LangChain, Llama index, and AI orchestration
33 - Securing embedding models
34 - Securing vector databases
35 - Monitoring and incident response

Conclusion

36 - Securing generative AI - Summary