Alex Gamble Perplexity .Intercom is using the Realtime API to power Fin Voice, their AI Agent for phone support. The new model follows call scripts better, executes function calls more consistently, and hallucinates less, helping provide human-quality AI support, 24/7.

🆕 Four updates to building agents with OpenAI: Agents SDK in TypeScript, a new RealtimeAgent feature for voice agents, Traces support for the Realtime API, and improvements to our speech-to-speech model.

The Agents SDK is now available in TypeScript and supports handoffs, guardrails, tracing, MCP, and other core agent primitives, just like the Python version.

It includes new support for human-in-the-loop approvals, allowing you to pause tool execution, serialize and store the agent state, approve or reject specific calls, and resume the agent run.

You can also build voice agents that run in the client or on your server with the new RealtimeAgent feature, powered by the Realtime API. Define them like text agents, including tool calls, handoffs, guardrails and with automatic audio and interruption handling. Get started here:

Next, the Traces dashboard now supports Realtime API sessions, letting you visualize voice agent runs, including audio input/output, tool invocations, and interruptions, whether created via the API or the Agents SDK. Here's @_agamble to show you how it works:

Finally, we’re improving the instruction following reliability, tool calling consistency, and interruption behavior of our speech-to-speech model, and introducing a new `speed` parameter in the API that lets you control how fast the voice speaks during each session.

The updated model is now available as gpt-4o-realtime-preview-2025-06-03 in the Realtime API and gpt-4o-audio-preview-2025-06-03 in the Chat Completions API. Here’s what early testers have to say about it:

.@perplexity_ai's voice mode uses the Realtime API to provide fast, accurate answers through natural voice interactions. They found the new model improved tool calling accuracy, resulting in more reliable and engaging conversations.

.@VolleyGames is developing a fantasy RPG featuring an AI dungeon master powered by the Realtime API. They found the latest model was better at following game rules while generating a more creative narrative, delivering a smoother and more engaging gameplay experience.

Hope these updates help you build even more useful voice agents! Please keep the feedback coming — we’re continuing to make more improvements to the Agents SDK and Realtime API. And BTW, if you’re at AI Engineer World’s Fair, join us this afternoon for more about these updates:

Stop wasting time following up with leads. Let our AI agents do it for you.

@_agamble @perplexity_ai @intercom B2b for OpenAI bringing the big millions. Fascinating to see real world implementation of AI

@_agamble @perplexity_ai @intercom Good job on the upgrades, but, I won't be impressed by any SOTA model until it can understand when I take a long breath or ponder on my next word mid sentence and NOT interrupt me. THAT is the next omg moment.

@_agamble @perplexity_ai @intercom the realtime API is too expensive tho