Loading video...

Video Failed to Load

Go Home

Next, the Traces dashboard now supports Realtime API sessions, letting you visualize voice agent runs, including audio input/output, tool invocations, and interruptions, whether created via the API or the Agents SDK. Here's Alex Gamble to show you how it works:

13,411 views • 1 year ago •via X (Twitter)

11 Comments

OpenAI Developers's profile picture
OpenAI Developers1 year ago

🆕 Four updates to building agents with OpenAI: Agents SDK in TypeScript, a new RealtimeAgent feature for voice agents, Traces support for the Realtime API, and improvements to our speech-to-speech model.

OpenAI Developers's profile picture
OpenAI Developers1 year ago

The Agents SDK is now available in TypeScript and supports handoffs, guardrails, tracing, MCP, and other core agent primitives, just like the Python version.

OpenAI Developers's profile picture
OpenAI Developers1 year ago

It includes new support for human-in-the-loop approvals, allowing you to pause tool execution, serialize and store the agent state, approve or reject specific calls, and resume the agent run.

OpenAI Developers's profile picture
OpenAI Developers1 year ago

You can also build voice agents that run in the client or on your server with the new RealtimeAgent feature, powered by the Realtime API. Define them like text agents, including tool calls, handoffs, guardrails and with automatic audio and interruption handling. Get started here:

OpenAI Developers's profile picture
OpenAI Developers1 year ago

Finally, we’re improving the instruction following reliability, tool calling consistency, and interruption behavior of our speech-to-speech model, and introducing a new `speed` parameter in the API that lets you control how fast the voice speaks during each session.

OpenAI Developers's profile picture
OpenAI Developers1 year ago

The updated model is now available as gpt-4o-realtime-preview-2025-06-03 in the Realtime API and gpt-4o-audio-preview-2025-06-03 in the Chat Completions API. Here’s what early testers have to say about it:

OpenAI Developers's profile picture
OpenAI Developers1 year ago

.@perplexity_ai's voice mode uses the Realtime API to provide fast, accurate answers through natural voice interactions. They found the new model improved tool calling accuracy, resulting in more reliable and engaging conversations.

OpenAI Developers's profile picture
OpenAI Developers1 year ago

@perplexity_ai .@intercom is using the Realtime API to power Fin Voice, their AI Agent for phone support. The new model follows call scripts better, executes function calls more consistently, and hallucinates less, helping provide human-quality AI support, 24/7.

OpenAI Developers's profile picture
OpenAI Developers1 year ago

.@VolleyGames is developing a fantasy RPG featuring an AI dungeon master powered by the Realtime API. They found the latest model was better at following game rules while generating a more creative narrative, delivering a smoother and more engaging gameplay experience.

OpenAI Developers's profile picture
OpenAI Developers1 year ago

Hope these updates help you build even more useful voice agents! Please keep the feedback coming — we’re continuing to make more improvements to the Agents SDK and Realtime API. And BTW, if you’re at AI Engineer World’s Fair, join us this afternoon for more about these updates:

Etherspot 🐞's profile picture
Etherspot 🐞2 years ago

Want to build a Web3 app without all the overhead costs? Try TransactionKit, a React toolkit that makes building dApps easy. Integrate cutting-edge account abstraction features in just a few clicks. Try it today 👇

Related Videos