Loading video...

Video Failed to Load

Go Home

Last week we released OpenAI Developers gpt-realtime our latest speech-to-speech model. Prompting S2S models can be extremely powerful. To get the most out of the model we compiled a series of prompting tips of what we saw work with early customers. To present them here's Cedar our new voice...

dkundel's profile picture

dominik kundel

6,068 subscribers

66,904 views • 11 months ago •via X (Twitter)

Science & TechnologyEducation
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)
3:13

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

55,222 views • 3 months ago

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.
4:03

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.

OpenAI

3,657,484 views • 3 months ago

the new OpenAI realtime voice model just released + gpt 5.5 fast mode brings us a new possibility - realtime speech to live presentation! i just talk, and the whiteboard would whiteboard itself prototype is open sourced. details in thread below -
1:46

the new OpenAI realtime voice model just released + gpt 5.5 fast mode brings us a new possibility - realtime speech to live presentation! i just talk, and the whiteboard would whiteboard itself prototype is open sourced. details in thread below -

Kun Chen

114,908 views • 3 months ago

The Realtime API is officially out of beta and ready for your production voice agents! We’re also introducing gpt-realtime—our most advanced speech-to-speech model yet—plus new voices and API capabilities: 🔌 Remote MCPs 🖼️ Image input 📞 SIP phone calling ♻️ Reusable prompts
1:40

The Realtime API is officially out of beta and ready for your production voice agents! We’re also introducing gpt-realtime—our most advanced speech-to-speech model yet—plus new voices and API capabilities: 🔌 Remote MCPs 🖼️ Image input 📞 SIP phone calling ♻️ Reusable prompts

OpenAI Developers

561,897 views • 11 months ago

Taking talking shop to a whole new level. We just shipped Glean’s real-time voice capability, powered by OpenAI’s newest speech model GPT-Realtime-2. Grounded in the context across your org, it feels like a real AI coworker and can keep up with how work gets finished. In internal evals, GPT-Realtime-2 delivered a 42.9% relative increase in helpfulness over its previous version. Give it a try. It speaks for itself. OpenAIDevs
1:55

Taking talking shop to a whole new level. We just shipped Glean’s real-time voice capability, powered by OpenAI’s newest speech model GPT-Realtime-2. Grounded in the context across your org, it feels like a real AI coworker and can keep up with how work gets finished. In internal evals, GPT-Realtime-2 delivered a 42.9% relative increase in helpfulness over its previous version. Give it a try. It speaks for itself. OpenAIDevs

Glean

16,588 views • 3 months ago

Prompting for Veo 3.1 just got easier. Our new prompting agent takes the guesswork out of getting great results. Check out our latest tutorial to learn how to get the most from Veo 3.1. Link below 👇
0:26

Prompting for Veo 3.1 just got easier. Our new prompting agent takes the guesswork out of getting great results. Check out our latest tutorial to learn how to get the most from Veo 3.1. Link below 👇

GLIF

18,551 views • 9 months ago

Over the past six months, we shipped 30+ models, features, and upgraded tools for the API. Our changelog has been busy. Here’s what you may have missed for the API: New models • GPT-5.5 • GPT-5.4 mini • GPT-5.4 nano • GPT-Realtime-2 • GPT-Realtime-Whisper • GPT-Realtime-Translate • GPT-Image-2 Agent building blocks • Agents SDK harness & sandbox • Skills in the Responses API • Server-side compaction • WebSocket mode • Hosted shell More ways to build • Admin API enhancements • OpenAI CLI • Models on Amazon Bedrock
2:06

Over the past six months, we shipped 30+ models, features, and upgraded tools for the API. Our changelog has been busy. Here’s what you may have missed for the API: New models • GPT-5.5 • GPT-5.4 mini • GPT-5.4 nano • GPT-Realtime-2 • GPT-Realtime-Whisper • GPT-Realtime-Translate • GPT-Image-2 Agent building blocks • Agents SDK harness & sandbox • Skills in the Responses API • Server-side compaction • WebSocket mode • Hosted shell More ways to build • Admin API enhancements • OpenAI CLI • Models on Amazon Bedrock

OpenAI Developers

158,800 views • 1 month ago

A lot of people shat on GPT-5 when it came out All of those people are wrong. GPT-5 is actually insanely good. It just requires a different style of prompting than any other model If you prompt it correctly, it's the best AI model out there Here's exactly how to do that:
21:52

A lot of people shat on GPT-5 when it came out All of those people are wrong. GPT-5 is actually insanely good. It just requires a different style of prompting than any other model If you prompt it correctly, it's the best AI model out there Here's exactly how to do that:

Alex Finn

19,286 views • 11 months ago

Our latest speech-to-speech model is faster, more accurate, and excels at function calling. Watch @promptshant and Brian Fioca build a realtime voice agent that can search the web and hand off tasks to reasoning models with full context.
1:58

Our latest speech-to-speech model is faster, more accurate, and excels at function calling. Watch @promptshant and Brian Fioca build a realtime voice agent that can search the web and hand off tasks to reasoning models with full context.

OpenAI Developers

81,822 views • 1 year ago

Some of our top customers are still choosing Llama 3.1 8B. For a while, we jumped to whatever hottest, latest model was taking up our twitter feed. 🙈 But as we are quickly realizing, to create a SOTA product, you need a model that fits your exact use case. Here’s what our customers tell us: > a lot of the legwork is actually around prompting > there’s an art to selecting and combining multiple models > benchmarks only show part of the picture. you have to understand the unique quirks of each model. Especially as model releases become more and more frequent, we need a clear way to evaluate new models. We have to break free of the naive trend to migrate to the ‘latest and greatest’. And you can easily achieve this using tools like Cerebras and Braintrust to swap models safely (without breaking production).
1:58

Some of our top customers are still choosing Llama 3.1 8B. For a while, we jumped to whatever hottest, latest model was taking up our twitter feed. 🙈 But as we are quickly realizing, to create a SOTA product, you need a model that fits your exact use case. Here’s what our customers tell us: > a lot of the legwork is actually around prompting > there’s an art to selecting and combining multiple models > benchmarks only show part of the picture. you have to understand the unique quirks of each model. Especially as model releases become more and more frequent, we need a clear way to evaluate new models. We have to break free of the naive trend to migrate to the ‘latest and greatest’. And you can easily achieve this using tools like Cerebras and Braintrust to swap models safely (without breaking production).

Cerebras

346,446 views • 8 months ago

"Civilization is the only example of super intelligence." Our CEO @davidpstout on Startup Project explaining why we're building millions of specialized models across devices — not one giant model in the cloud. A few highlights: – Why prompting loses money – 8B models matching GPT-3.5 – How we beat GPT-5 by 15% on real data Listen to the full episode below 👇
1:13

"Civilization is the only example of super intelligence." Our CEO @davidpstout on Startup Project explaining why we're building millions of specialized models across devices — not one giant model in the cloud. A few highlights: – Why prompting loses money – 8B models matching GPT-3.5 – How we beat GPT-5 by 15% on real data Listen to the full episode below 👇

webAI

325,138 views • 10 months ago

Generalist CEO Pete Florence says robotics models are in a transition period similar to the step change between GPT-2 and GPT-3. They're "starting to cross over into levels of performance where these things are commercially viable for a number of different applications." "We think this is a crossover point where we have a general model starting to be able to hit levels of reliability, speed, and improvisational intelligence where we can start to get these things out there." "Very much like — you take a GPT-2-level model, you scale it to a GPT-3-level model, and certain types of commercial applications start to become viable."
2:53

Generalist CEO Pete Florence says robotics models are in a transition period similar to the step change between GPT-2 and GPT-3. They're "starting to cross over into levels of performance where these things are commercially viable for a number of different applications." "We think this is a crossover point where we have a general model starting to be able to hit levels of reliability, speed, and improvisational intelligence where we can start to get these things out there." "Very much like — you take a GPT-2-level model, you scale it to a GPT-3-level model, and certain types of commercial applications start to become viable."

TBPN

23,356 views • 2 months ago

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.
0:37

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.

ElevenLabs

113,343 views • 2 years ago

More tests of painting with a realtime ai transformation. This time, the model settings ensure that the ai stays closer to the painting, which gives me more control over the final look of things. Eventually, we should be able to get as fine-grained control as we want with our painting.. to be able to put most of our energy into painting details in areas that matter the most in the composition. I'll post below how I got this to work. #painting #ai
0:45

More tests of painting with a realtime ai transformation. This time, the model settings ensure that the ai stays closer to the painting, which gives me more control over the final look of things. Eventually, we should be able to get as fine-grained control as we want with our painting.. to be able to put most of our energy into painting details in areas that matter the most in the composition. I'll post below how I got this to work. #painting #ai

Martin Nebelong

594,503 views • 2 years ago

As models get smarter and more capable, you will get diminishing returns prompting them yourself. Enter "Orchestrate" a powerful new workflow in Repo Prompt that allows you to combine the strengths of GPT 5.5 and Opus 4.7 in ways no other tool makes possible.
14:17

As models get smarter and more capable, you will get diminishing returns prompting them yourself. Enter "Orchestrate" a powerful new workflow in Repo Prompt that allows you to combine the strengths of GPT 5.5 and Opus 4.7 in ways no other tool makes possible.

eric provencher

67,148 views • 3 months ago

Voice-only programming with the new OpenAI Realtime API ... I spend a lot of time these days pair programming with LLMs. Often I'm talking rather than typing. This "voice dictation" use case has become an important vibe benchmark for me. Being able to create text input just by talking, flexibly, in a context dependent way, with tool calling, is a *hard* problem for today's models. Natural language dictation requires a very high degree of contextual intelligence, instruction following accuracy, and tool calling reliability. Today's new gpt-realtime model is quite good at this hard problem. The original realtime model release last year was impressive. Seeing what a speech-to-speech model could do got a lot of people excited about the possibilities of voice AI. The improvements since that first release are equally impressive. I can use this new model, now, for real world tasks that were past the edge of the "jagged frontier" before. Here's a video showing a couple of fun (and tricky) modes of voice input.
2:40

Voice-only programming with the new OpenAI Realtime API ... I spend a lot of time these days pair programming with LLMs. Often I'm talking rather than typing. This "voice dictation" use case has become an important vibe benchmark for me. Being able to create text input just by talking, flexibly, in a context dependent way, with tool calling, is a *hard* problem for today's models. Natural language dictation requires a very high degree of contextual intelligence, instruction following accuracy, and tool calling reliability. Today's new gpt-realtime model is quite good at this hard problem. The original realtime model release last year was impressive. Seeing what a speech-to-speech model could do got a lot of people excited about the possibilities of voice AI. The improvements since that first release are equally impressive. I can use this new model, now, for real world tasks that were past the edge of the "jagged frontier" before. Here's a video showing a couple of fun (and tricky) modes of voice input.

kwindla

50,874 views • 11 months ago

Today we launched Eleven v3 to alpha. It's our most expressive and dynamic speech model to date. Check out the landing page which showcases some of the incredible, creative outputs that the model is capable of.
0:37

Today we launched Eleven v3 to alpha. It's our most expressive and dynamic speech model to date. Check out the landing page which showcases some of the incredible, creative outputs that the model is capable of.

Nev Flynn

10,311 views • 1 year ago

Databricks is excited to partner with OpenAI on GPT-5.5, their latest frontier model. GPT-5.5 will be available in Unity AI Gateway on launch. You can use it with coding tools such as Codex, or to power your enterprise agents. GPT-5.5 is state-of-the-art on many benchmarks including OfficeQA Pro, our benchmark for evaluating grounded reasoning on enterprise tasks. We are partnering with OpenAI to co-launch on Databricks. Hear more from our co-founder Patrick Wendell and OpenAI CRO Denise Holland Dresser on GPT-5.5 in Databricks.
1:53

Databricks is excited to partner with OpenAI on GPT-5.5, their latest frontier model. GPT-5.5 will be available in Unity AI Gateway on launch. You can use it with coding tools such as Codex, or to power your enterprise agents. GPT-5.5 is state-of-the-art on many benchmarks including OfficeQA Pro, our benchmark for evaluating grounded reasoning on enterprise tasks. We are partnering with OpenAI to co-launch on Databricks. Hear more from our co-founder Patrick Wendell and OpenAI CRO Denise Holland Dresser on GPT-5.5 in Databricks.

Databricks

12,707 views • 3 months ago

Fascinating to hear how openly Krishna talked about the trade-offs around how Anthropic allocates compute: "If you buy too much compute, you go out of business. If you buy too little, you can't serve your customers and you're not at the frontier. Same thing." "There's a level of compute for model development we will not go below, even if it means it's harder to serve customers." "If we said to our employees, you can't use our models anymore, we could serve billions of dollars of revenue with that compute." "But we want to take a long-term view [because] the returns to frontier intelligence are extremely high, especially in enterprise."
2:08

Fascinating to hear how openly Krishna talked about the trade-offs around how Anthropic allocates compute: "If you buy too much compute, you go out of business. If you buy too little, you can't serve your customers and you're not at the frontier. Same thing." "There's a level of compute for model development we will not go below, even if it means it's harder to serve customers." "If we said to our employees, you can't use our models anymore, we could serve billions of dollars of revenue with that compute." "But we want to take a long-term view [because] the returns to frontier intelligence are extremely high, especially in enterprise."

Patrick OShaughnessy

83,688 views • 2 months ago

As we walk into a new week, let this be the moment we stop second guessing ourselves. May this be the beginning of trusting our guts, of listening to our instincts. As a new week begins, believe in yourself and all that you have done to get yourself to this place. Keep going, keep moving forward, keep at it! #sundayeveningreminder
0:21

As we walk into a new week, let this be the moment we stop second guessing ourselves. May this be the beginning of trusting our guts, of listening to our instincts. As a new week begins, believe in yourself and all that you have done to get yourself to this place. Keep going, keep moving forward, keep at it! #sundayeveningreminder

Maria Shriver

11,425 views • 5 months ago