正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Voice is the OG modality. So excited for image inputs, function calling & MCP support in the Realtime API GA! `gpt-realtime` is a lot more natural and expressive, and every time a SOTA voice model is released, you know what I gotta do... Here is the new voice Marin,... show more

swyx

127,338 subscribers

51,219 次观看 • 10 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.

OpenAI

3,618,936 次观看 • 1 个月前

The Realtime API is officially out of beta and ready for your production voice agents! We’re also introducing gpt-realtime—our most advanced speech-to-speech model yet—plus new voices and API capabilities: 🔌 Remote MCPs 🖼️ Image input 📞 SIP phone calling ♻️ Reusable prompts

The Realtime API is officially out of beta and ready for your production voice agents! We’re also introducing gpt-realtime—our most advanced speech-to-speech model yet—plus new voices and API capabilities: 🔌 Remote MCPs 🖼️ Image input 📞 SIP phone calling ♻️ Reusable prompts

OpenAI Developers

561,678 次观看 • 10 个月前

Introducing Realtime TTS-2, a new generation of voice model built for realtime conversation. It is the first voice model that hears the conversation, takes natural-language voice direction, holds one voice identity across over 100 languages, and speaks like a person who is paying attention. The result is voice AI that feels as good as it sounds. Try it out: Learn More:

Introducing Realtime TTS-2, a new generation of voice model built for realtime conversation. It is the first voice model that hears the conversation, takes natural-language voice direction, holds one voice identity across over 100 languages, and speaks like a person who is paying attention. The result is voice AI that feels as good as it sounds. Try it out: Learn More:

Inworld AI

324,372 次观看 • 1 个月前

Voice is an interface, and I feel we’re going to see voice explode. OpenAI gpt-realtime is out and allows for native speech-to-speech and precise tool calling in real time. It can also take image inputs and place SIP phone calls - good for a quick triage before human handoff.

Voice is an interface, and I feel we’re going to see voice explode. OpenAI gpt-realtime is out and allows for native speech-to-speech and precise tool calling in real time. It can also take image inputs and place SIP phone calls - good for a quick triage before human handoff.

Allie K. Miller

79,857 次观看 • 9 个月前

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

54,912 次观看 • 1 个月前

Voice workflows just got stronger with gpt-realtime-1.5 in the Realtime API. The model offers more reliable instruction following, tool calling, and multilingual accuracy. Demo with Charlie Guo

Voice workflows just got stronger with gpt-realtime-1.5 in the Realtime API. The model offers more reliable instruction following, tool calling, and multilingual accuracy. Demo with Charlie Guo

OpenAI Developers

417,428 次观看 • 4 个月前

the new OpenAI realtime voice model just released + gpt 5.5 fast mode brings us a new possibility - realtime speech to live presentation! i just talk, and the whiteboard would whiteboard itself prototype is open sourced. details in thread below -

the new OpenAI realtime voice model just released + gpt 5.5 fast mode brings us a new possibility - realtime speech to live presentation! i just talk, and the whiteboard would whiteboard itself prototype is open sourced. details in thread below -

Kun Chen

114,160 次观看 • 1 个月前

OpenAI's o1 model is officially out of preview and available to developers via API! Plus, new capabilities for Realtime API builders (like the cool stuffed animal voice demo below). New o1 features include: > Function calling to connect with external data > Developer messages for tone/style control > Image analysis capabilities > "reasoning_effort" to control thinking time > Preference fine-tuning This is a huge unlock for developers (but the pricing is not cheap). $15 per ~750k words analyzed and $60 per ~750k words generated — roughly 3-4x more than GPT-4o. In addition to the o1 news, we also got updates to the Realtime API: > WebRTC integration (smoother voice apps) > 60% price drop for GPT-4o audio > New GPT-4o and 4o mini models > Background task support Big day for AI builders!

OpenAI's o1 model is officially out of preview and available to developers via API! Plus, new capabilities for Realtime API builders (like the cool stuffed animal voice demo below). New o1 features include: > Function calling to connect with external data > Developer messages for tone/style control > Image analysis capabilities > "reasoning_effort" to control thinking time > Preference fine-tuning This is a huge unlock for developers (but the pricing is not cheap). $15 per ~750k words analyzed and $60 per ~750k words generated — roughly 3-4x more than GPT-4o. In addition to the o1 news, we also got updates to the Realtime API: > WebRTC integration (smoother voice apps) > 60% price drop for GPT-4o audio > New GPT-4o and 4o mini models > Background task support Big day for AI builders!

Rowan Cheung

38,690 次观看 • 1 年前

just found out from Alex Volkov's show that @chatgptapp advanced voice upgraded singing capability this week so here is karaokebench would say it is like 3/7 so far are there other sota singing models?

just found out from Alex Volkov's show that @chatgptapp advanced voice upgraded singing capability this week so here is karaokebench would say it is like 3/7 so far are there other sota singing models?

swyx 🇸🇬 AIE Singapore!

71,555 次观看 • 1 年前

Alex Gamble Perplexity .Intercom is using the Realtime API to power Fin Voice, their AI Agent for phone support. The new model follows call scripts better, executes function calls more consistently, and hallucinates less, helping provide human-quality AI support, 24/7.

Alex Gamble Perplexity .Intercom is using the Realtime API to power Fin Voice, their AI Agent for phone support. The new model follows call scripts better, executes function calls more consistently, and hallucinates less, helping provide human-quality AI support, 24/7.

OpenAI Developers

23,409 次观看 • 1 年前

Voice-only programming with the new OpenAI Realtime API ... I spend a lot of time these days pair programming with LLMs. Often I'm talking rather than typing. This "voice dictation" use case has become an important vibe benchmark for me. Being able to create text input just by talking, flexibly, in a context dependent way, with tool calling, is a *hard* problem for today's models. Natural language dictation requires a very high degree of contextual intelligence, instruction following accuracy, and tool calling reliability. Today's new gpt-realtime model is quite good at this hard problem. The original realtime model release last year was impressive. Seeing what a speech-to-speech model could do got a lot of people excited about the possibilities of voice AI. The improvements since that first release are equally impressive. I can use this new model, now, for real world tasks that were past the edge of the "jagged frontier" before. Here's a video showing a couple of fun (and tricky) modes of voice input.

Voice-only programming with the new OpenAI Realtime API ... I spend a lot of time these days pair programming with LLMs. Often I'm talking rather than typing. This "voice dictation" use case has become an important vibe benchmark for me. Being able to create text input just by talking, flexibly, in a context dependent way, with tool calling, is a hard problem for today's models. Natural language dictation requires a very high degree of contextual intelligence, instruction following accuracy, and tool calling reliability. Today's new gpt-realtime model is quite good at this hard problem. The original realtime model release last year was impressive. Seeing what a speech-to-speech model could do got a lot of people excited about the possibilities of voice AI. The improvements since that first release are equally impressive. I can use this new model, now, for real world tasks that were past the edge of the "jagged frontier" before. Here's a video showing a couple of fun (and tricky) modes of voice input.

kwindla

50,806 次观看 • 10 个月前

Built a "YouTube realtime copilot" browser extension using OpenAI's realtime 2 API: The agent watches the video alongside you, and can answer any question you have about what was just said via realtime voice chat. The crazy part to me is: It can differentiate the YouTube's audio stream and your voice, so it doesn't confuse the video as commands, and stays silent unless you ask something!

Built a "YouTube realtime copilot" browser extension using OpenAI's realtime 2 API: The agent watches the video alongside you, and can answer any question you have about what was just said via realtime voice chat. The crazy part to me is: It can differentiate the YouTube's audio stream and your voice, so it doesn't confuse the video as commands, and stays silent unless you ask something!

Zara Zhang

44,631 次观看 • 1 个月前

My favorite demo of the new gpt-realtime model from Matthieu LC -- Shoggoth Mini using Realtime API with image input

My favorite demo of the new gpt-realtime model from Matthieu LC -- Shoggoth Mini using Realtime API with image input

Peter Bakkum

67,777 次观看 • 10 个月前

The new Realtime API from OpenAI is incredible... Watch it order 400 strawberries by actually CALLING the store with twillio. All with voice. 🍓🎤

The new Realtime API from OpenAI is incredible... Watch it order 400 strawberries by actually CALLING the store with twillio. All with voice. 🍓🎤

Ty

495,453 次观看 • 1 年前

That’s it. That’s the tweet. The Realtime API now supports WebRTC—you can add Realtime capabilities with just a handful of lines of code. We’ve also cut prices by 60%, added GPT-4o mini (10x cheaper than previous prices), improved voice quality, and made inputs more reliable.

That’s it. That’s the tweet. The Realtime API now supports WebRTC—you can add Realtime capabilities with just a handful of lines of code. We’ve also cut prices by 60%, added GPT-4o mini (10x cheaper than previous prices), improved voice quality, and made inputs more reliable.

OpenAI Developers

523,470 次观看 • 1 年前

Our latest speech-to-speech model is faster, more accurate, and excels at function calling. Watch @promptshant and Brian Fioca build a realtime voice agent that can search the web and hand off tasks to reasoning models with full context.

Our latest speech-to-speech model is faster, more accurate, and excels at function calling. Watch @promptshant and Brian Fioca build a realtime voice agent that can search the web and hand off tasks to reasoning models with full context.

OpenAI Developers

81,822 次观看 • 1 年前

Developers are building great voice experiences with the Realtime API. See how Lemonade uses it to power AI Maya, their friendly and engaging voice agent. The Realtime API's automatic voice detection and low latency enable them to offer 24/7 multilingual phone support 📞

Developers are building great voice experiences with the Realtime API. See how Lemonade uses it to power AI Maya, their friendly and engaging voice agent. The Realtime API's automatic voice detection and low latency enable them to offer 24/7 multilingual phone support 📞

OpenAI Developers

126,086 次观看 • 1 年前

We're adding MCP capability to Realtime API, I'm very excited about how well MCP tools work over voice. Here's a demo using a Notion MCP --

We're adding MCP capability to Realtime API, I'm very excited about how well MCP tools work over voice. Here's a demo using a Notion MCP --

Peter Bakkum

27,261 次观看 • 10 个月前

The new Realtime API with web crawling is mind-blowing! Talk in realtime with any website. Powered by the OpenAI Realtime API and 🔥 Check it out:

The new Realtime API with web crawling is mind-blowing! Talk in realtime with any website. Powered by the OpenAI Realtime API and 🔥 Check it out:

Nicolas Camara

347,089 次观看 • 1 年前

Voice Design v3 is here. Create any voice you can imagine with a prompt. We’ve rebuilt the underlying Voice Design model to deliver higher quality and broader expressive range. Generate production-ready voices in 70+ languages with support for hundreds of localized accents.

Voice Design v3 is here. Create any voice you can imagine with a prompt. We’ve rebuilt the underlying Voice Design model to deliver higher quality and broader expressive range. Generate production-ready voices in 70+ languages with support for hundreds of localized accents.

ElevenLabs

153,848 次观看 • 1 年前