OpenAI API: Realtime API on the Server
40mGeneral2025-02-18
Authors
Morten Rand-Hendriksen
Senior Staff Instructor, Speaker, Web Designer, and Software Developer
Course details
With OpenAI’s Realtime API, you can build custom server-side real-time voice-to-voice AI chat into any application using WebSockets. In this course you’ll explore server-side integration of the Realtime API for tighter security and more advanced functionality. The course includes a hands-on example building on the WebRTC projects from the previous course, and covers how to capture and record user audio, send and receive audio and text, and integrate the server-side API app with a headless front-end app using vanilla JavaScript.
Learning objectives
Discover how to set up a server environment for accessing the Realtime API.
Discover how to capture and receive audio from a headless app and send it to the server.
Explore how WebSockets sends and receives audio and text from the API.
Learning objectives
Discover how to set up a server environment for accessing the Realtime API.
Discover how to capture and receive audio from a headless app and send it to the server.
Explore how WebSockets sends and receives audio and text from the API.
Skills covered
OpenAI APIAPIsOpenAIGenerative AIArtificial Intelligence (AI)Software DevelopmentOne-Off
Concepts
0. Introduction
- 01 - Voice-to-voice chat over WebSockets
- 02 - Prerequisites, references, and exercise files
1. Realtime API - Websockets Edition
- 03 - Comparing approaches - WebSockets and WebRTC
- 04 - Setup and walkthrough
- 05 - Creating a WebSocket connection to the Realtime API
- 06 - Sending and receiving events
- 07 - The Realtime API reference
- 08 - Sending and receiving audio messages
- 09 - Function calling
2. Conclusion
- 10 - Next steps