Creating Agents with Python and the AI Toolkit for Visual Studio Code

Course details

Curious about how to build AI agents using Python? In this course, AI engineer April Gittens shows you how to create intelligent, tool-using agents with the AI Toolkit extension for Visual Studio Code. Build your understanding of agent architecture, writing effective prompts, and using schemas for structured outputs. Then enhance your agent with custom tools, evaluate its responses with built-in metrics, and prepare it for real-world scenarios. Whether you're building a productivity assistant, a research bot, or a creative companion, this course will give you the hands-on skills to bring AI agents to life—all within your existing Python workflow.

Learning objectives
Describe the core components of an AI agent and the features of the AI Toolkit extension for Visual Studio Code.
Create and configure Python-based AI agents using the Agent Builder and schema-based output design.
Develop and configure custom tools to extend your agent's capabilities with real-world functionality.
Evaluate and refine agent performance using built-in metrics such as relevance, coherence, and fluency.

Concepts

Introduction

Why AI agents matter

Introduction to the AI Toolkit for Visual Studio Code

Installing and configuring the extension in Visual Studio Code
Overview of the AI Toolkit panels and commands

Choosing and Using Models

Exploring the Model Catalog
Find a model for your scenario
Using the Playground to test prompts and analyze outputs
Deploying models from Azure or using hosted endpoints

Creating AI Agents with Agent Builder

Creating your first agent in the Agent Builder
Writing effective system prompts
Understanding and modifying schemas for structured output
Running and testing your agent
Agent lifecycle and behavior flows

Enhancing Agents with Tools

What are tools in the context of an AI agent
Understanding Model Context Protocol (MCP)
Enabling tool-use in your agent's configuration
Creating a custom MCP server

Evaluating Agent Responses

Introduction to agent evaluation
Generating a synthetic dataset
Run manual evaluations
Run automated evaluations

Agent Orchestration with Code

Migrating your agent to a standalone Python workflow
Review the agent code
Prepare the environment to run the agent code
Run the agent code

Observability and Debugging for AI Agents

Understanding why observability matters for AI agents
Logging agent inputs, outputs, and tool usage
Using trace view to inspect step-by-step agent behavior

Conclusion

Next steps on your learning journey