OpenAI Realtime API: Building Voice Agents with Realtime API and the Agents SDK

Build voice-to-voice, text-to-voice, and image-to-voice AI agents with OpenAI’s Realtime API and the Agents SDK. In this course, you’ll explore how the Realtime API works, how to build realtime voice agent apps using the Agents SDK, and how to enable advanced features including guardrails, function calling, MCP servers, and live image input via device camera. You’ll also learn how to authenticate front-end apps using a custom auth server and ephemeral tokens.

Learning objectives
Deploy an auth server to authenticate front-end and other apps using ephemeral tokens.
Use OpenAI Agents SDK to build realtime voice apps.
Understand how server transport events work and how to use them to manage realtime WebRTC sessions.
Add live text transcripts for user input and AI output to improve accessibility.
Add a text-input option for text-based chat.
Configure guardrails to prevent the AI from outputting specified words and phrases.
Develop custom functions and include them as tools for the agent.
Build custom sub-agents.