OpenAI API: Building Voice Apps with the Realtime API (October 2024)
53mGeneral2024-10-10
Authors
Morten Rand-Hendriksen
Senior Staff Instructor, Speaker, Web Designer, and Software Developer
Course details
The Realtime API from OpenAI gives you the tools and materials needed to integrate latency-free voice-to-voice interaction in your AI apps while leveraging existing features like function calling, memory, and more. In this course you’ll get an introduction to the Realtime API, a rundown of the documentation and provided examples, and hands-on practice building your own advanced voice- and text-powered AI app with function calling.
Learning objectives
Understand what the Realtime API is and what it can do.
Use the Realtime API to activate voice-to-voice interaction.
Configure the Realtime API with different voices and different instructions.
Build a React app with voice features and server-based AI integration for security.
Learning objectives
Understand what the Realtime API is and what it can do.
Use the Realtime API to activate voice-to-voice interaction.
Configure the Realtime API with different voices and different instructions.
Build a React app with voice features and server-based AI integration for security.
Skills covered
OpenAI APITelecommunicationsAPIsOpenAIFull-Stack Web DevelopmentGenerative AIArtificial Intelligence (AI)Web DevelopmentNetwork and System AdministrationSoftware DevelopmentOne-Off
Concepts
0. Introduction
- 01 - Add voice-to-voice interaction to any app
- 02 - Caution - Voice costs and spend limits
1. Voice Chat and the Realtime API
- 03 - Introducing the Realtime API
- 04 - Testing the Realtime API in the Playground
- 05 - Configuring the Realtime API's responses
- 06 - Documentation and examples
2. Exploring the Realtime API
- 07 - Introducing the OpenAI Realtime Console
- 08 - Setting up the Realtime Console
- 09 - Exploring the API through the console
- 10 - Using a relay server for secure key handling
- 11 - Voice-enabled function calling
- 12 - Getting the system to store information
- 13 - Configuring voice and instructions
- 14 - Realtime API - Key concepts
- 15 - The Azure Realtime audio SDK
Conclusion
- 16 - Tips for talking to machines