Securing Generative AI: Strategies, Methodologies, Tools, and Best Practices

Course details

This course offers a comprehensive exploration into the crucial security measures necessary for the deployment and development of various AI implementations, including large language models (LLMs) and retrieval-augmented generation (RAG). Discover key considerations and mitigations to reduce the overall risk in organizational AI system development processes. Author and tech trainer Omar Santos covers the essentials of secure-by-design principles, focusing on security outcomes, radical transparency, and building organizational structures that prioritize security. Along the way, learn more about AI threats, LLM security, prompt injection, insecure output handling, red team AI models, and more. By the end of this course, you’ll be prepared to wield your newly honed skills to protect RAG implementations, secure vector databases, select embedding models, and leverage powerful orchestration libraries like LangChain and LlamaIndex.

Learning objectives
Explore security for deploying and developing AI applications, retrieval-augmented generation (RAG), agents, and other AI implementations.
Leverage hands-on practical skills drawn from real-world AI and machine learning cases.
Incorporate key security considerations at every stage of AI development, deployment, and operation.

Concepts

Introduction

Securing generative AI - Introduction

Introduction to AI Threats and LLM Security

Learning objectives
Understanding the significance of LLMs in the AI landscape
Exploring the resources for this course - GitHub repositories and others
Introducing retrieval augmented generation (RAG)
Understanding the OWASP Top 10 risks for LLMs
Exploring the MITRE ATLAS (adversarial threat landscape for artificial intelligence systems) framework
Understanding the NIST taxonomy and terminology of attacks and mitigations

Understanding Prompt Injection Insecure Output Handling

Learning objectives
Defining prompt injection attacks
Exploring real-life prompt injection attacks
Using ChatML for OpenAI API calls to indicate to the LLM the source of prompt input
Enforcing privilege control on LLM access to back-end systems
Best practices around API tokens for plugins, data access, and function-level permissions
Understanding insecure output handling attacks
Using the OWASP ASVS to protect against insecure output handling

Training Data Poisoning, Model Denial of Service Supply Chain Vulnerabilities

Learning objectives
Understanding training data poisoning attacks
Exploring model denial of service attacks
Understanding the risks of the AI and ML supply chain
Best practices when using open-source models from Hugging face and other sources
Securing Amazon Bedrock, Amazon SageMaker, Microsoft Azure AI services, and other environments

Sensitive Information Disclosure, Insecure Plugin Design, and Excessive Agency

Learning objective
Understanding sensitive information disclosure
Exploiting insecure plugin design
Avoiding excessive agency

Overreliance, Model Theft, and Red Teaming AI Models

Learning objectives
Understanding overreliance
Exploring model theft attacks
Understanding red teaming of AI models

Protecting Retrieval Augmented Generation (RAG) Implementations

Learning objectives
Understanding RAG, LangChain, Llama index, and AI orchestration
Securing embedding models
Securing vector databases
Monitoring and incident response

Conclusion

Securing generative AI - Summary