Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

hey everyone! super excited play a small role in the OpenAI Realtime API launch today. we have launched an open source repo, the Realtime Console -- a React app you can hack apart and start building with immediately. here's a demo! thanks Peter Bakkum Mark Hudnall Jordan Sitkin

keith

5,571 subscribers

243,226 views • 1 year ago •via X (Twitter)

Education Comedy Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

keith1 year ago

GitHub repo here:

Emil Eberhard1 year ago

@OpenAI @pbbakkum @landakram @DustMason "Could not connect to "wss://api.openai.com/v1/realtime"" ^ is this because it's not online yet?

keith1 year ago

@OpenAI @pbbakkum @landakram @DustMason rolling out throughout the week!

Jake1 year ago

@OpenAI @pbbakkum @landakram @DustMason If you wanna play with this, you can 1-Click deploy it on Railway! Use the dashboard demo’d today, plus the we socket API

Mervin Praison1 year ago

@OpenAI @pbbakkum @landakram @DustMason @keithwhor I am getting this error.

svapnil1 year ago

@OpenAI @pbbakkum @landakram @DustMason Is the Realtime API live for developers to test using their API Keys? I keep running into an error connecting to the websocket using the starter project!

keith1 year ago

@OpenAI @pbbakkum @landakram @DustMason

Daniel Nguyen ⚡1 year ago

@OpenAI @pbbakkum @landakram @DustMason Great work. Thank you

Atlas3D1 year ago

@OpenAI @pbbakkum @landakram @DustMason fantastic work! ty so much!

Boyan1 year ago

@OpenAI @pbbakkum @landakram @DustMason Just tried but it is expensive... Boyan on X: "Talking to #OpenAI Realtime Console is quite impressive. However, it is quite costly. Few minutes of test talk cost me $8.30, not nice but hoping it will come down to reasonable level. / X

Related Videos

At OpenAI DevDay, we introduced the Realtime API with a demo of an interactive solar system you can navigate with your voice. A lot of you have asked how it was built, so we've open-sourced it—with a twist: this version uses our new WebRTC integration. Have fun exploring! 🌍

At OpenAI DevDay, we introduced the Realtime API with a demo of an interactive solar system you can navigate with your voice. A lot of you have asked how it was built, so we've open-sourced it—with a twist: this version uses our new WebRTC integration. Have fun exploring! 🌍

Katia Gil Guzman

316,561 views • 1 year ago

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵

Craig Dennis

60,122 views • 1 year ago

Who knew that inviting AI into your video chats could be fun && functional? We used Cloudflare Developers Calls and the OpenAI Realtime API with WebRTC to add AI to our open source Orange Meets demo app. I forked it to make my own murder mystery app 😈 Code and details in 🧵

Who knew that inviting AI into your video chats could be fun && functional? We used Cloudflare Developers Calls and the OpenAI Realtime API with WebRTC to add AI to our open source Orange Meets demo app. I forked it to make my own murder mystery app 😈 Code and details in 🧵

Ricky

16,796 views • 1 year ago

🎶 Lyria RealTime is a new experimental interactive music generation model that allows anyone to interactively create, control and perform music in real time. Available via the Gemini API and you can try the demo app on Google AI Studio.

🎶 Lyria RealTime is a new experimental interactive music generation model that allows anyone to interactively create, control and perform music in real time. Available via the Gemini API and you can try the demo app on Google AI Studio.

Google AI Developers

25,739 views • 1 year ago

Want to build on the new Google Multimodal Live API with Gemini 2.0? My teammates Kyle Phillips + @tinaz0ne + Trudy Painter made open-source starter demos! 🧵 Here's a React.js boilerplate console you can start with. Code here:

Want to build on the new Google Multimodal Live API with Gemini 2.0? My teammates Kyle Phillips + @tinaz0ne + Trudy Painter made open-source starter demos! 🧵 Here's a React.js boilerplate console you can start with. Code here:

Alexander Chen

25,037 views • 1 year ago

Today we’re excited to introduce GenSX, an open source TypeScript framework for building backend workflows and agents with reusable, React-like components. If you know how to write a React component, then you’ll feel right at home building agents in GenSX.

Today we’re excited to introduce GenSX, an open source TypeScript framework for building backend workflows and agents with reusable, React-like components. If you know how to write a React component, then you’ll feel right at home building agents in GenSX.

Evan Boyle

14,446 views • 1 year ago

Built a small research playground with Nimble API. You enter a prompt: → Research OpenAI competitors → Find AI startups in (Any country of your choice) → Analyze the AI coding agent market The app turns it into a structured research report with summaries, findings, and source cards. Here's the workflow 👇

Built a small research playground with Nimble API. You enter a prompt: → Research OpenAI competitors → Find AI startups in (Any country of your choice) → Analyze the AI coding agent market The app turns it into a structured research report with summaries, findings, and source cards. Here's the workflow 👇

Kaitee

35,212 views • 1 month ago

We worked with OpenAI to build a native integration for the OpenAI Agents SDK. Today, we are very excited to launch as a tracing processor. With just a few lines of code, you can trace all your agent workflows and debug them much faster. Check out this quick demo to get started:

We worked with OpenAI to build a native integration for the OpenAI Agents SDK. Today, we are very excited to launch as a tracing processor. With just a few lines of code, you can trace all your agent workflows and debug them much faster. Check out this quick demo to get started:

Respan

58,055 views • 1 year ago

VoiceClaw Realtime will be available in the App Store within hours. Thanks again to everyone who tested the final beta over the weekend. Here's just a tiny bit of what it can do, directly on your phone, and on your computer via OpenClaw, from anywhere.

VoiceClaw Realtime will be available in the App Store within hours. Thanks again to everyone who tested the final beta over the weekend. Here's just a tiny bit of what it can do, directly on your phone, and on your computer via OpenClaw, from anywhere.

Ben Badejo

171,015 views • 2 months ago

OPENAI 🔥: A Realtime Voice Mode on Codex will trigger a Pet or an Orb to appear! > Users will be able to invoke them with the "Hey Chat" command. > The Orb mentioned in the Realtime Voice settings is likely the same Orb we see on ChatGPT today. > Additionally, Codex will get a Library section in the side nav, the same section we see on ChatGPT. Codex = ChatGPT soon 👀 * The video shows that Pet has been summoned via the Voice Mode button.

OPENAI 🔥: A Realtime Voice Mode on Codex will trigger a Pet or an Orb to appear! > Users will be able to invoke them with the "Hey Chat" command. > The Orb mentioned in the Realtime Voice settings is likely the same Orb we see on ChatGPT today. > Additionally, Codex will get a Library section in the side nav, the same section we see on ChatGPT. Codex = ChatGPT soon 👀 * The video shows that Pet has been summoned via the Voice Mode button.

🚨 AI News | TestingCatalog

68,831 views • 1 month ago

I'm building an AI app that lets you get deep work done. You share your screen, start a focus session, & get deep work done with real accountability. Will launch & open source the code in 24 hours.

I'm building an AI app that lets you get deep work done. You share your screen, start a focus session, & get deep work done with real accountability. Will launch & open source the code in 24 hours.

Hassan

15,745 views • 3 months ago

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

54,912 views • 2 months ago

Problem: After working for a few months in open source, I noticed that understanding issues is often very tough, and random OS files make it a nightmare for newbies. Solution: So here we go I created It explains any open-source repo file line by line with flowcharts, explains open-source issues, and provides a dedicated step-by-step solution plan. It also helps you write PR content in multiple languages. Built for the Apify × Lingo.dev Multilingual Actors Hackathon You can start using it by adding a Gemini or Groq API key. For more detailed explanations and better data fetching, you can add an Apify API key. For in-app translations, it uses Lingo. Even if you don’t believe in the app, you can check the repo code or fork it and deploy it yourself (SQL query provided) and use it unlimited times. Enjoy. Your Problem solved.

Problem: After working for a few months in open source, I noticed that understanding issues is often very tough, and random OS files make it a nightmare for newbies. Solution: So here we go I created It explains any open-source repo file line by line with flowcharts, explains open-source issues, and provides a dedicated step-by-step solution plan. It also helps you write PR content in multiple languages. Built for the Apify × Lingo.dev Multilingual Actors Hackathon You can start using it by adding a Gemini or Groq API key. For more detailed explanations and better data fetching, you can add an Apify API key. For in-app translations, it uses Lingo. Even if you don’t believe in the app, you can check the repo code or fork it and deploy it yourself (SQL query provided) and use it unlimited times. Enjoy. Your Problem solved.

Sk Akram

30,264 views • 7 months ago

Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument. MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency. Open weights. Open source inference engine. Suite of apps and plugins. Hear what it can do and try it out for yourself below 🧵

Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument. MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency. Open weights. Open source inference engine. Suite of apps and plugins. Hear what it can do and try it out for yourself below 🧵

Google Magenta Project

112,723 views • 1 month ago

here is the first look at realtime voice mode in codex the ui is already in the app, but the codex backend endpoint does not seem to be live yet i got it running by forcing the local feature gate and using my own openai api key, so it goes through the public realtime api instead it's incredible, but this demo alone cost me ~$4 gpt realtime 2 is very expensive, which makes me wonder how they're going to handle it - part of the normal usage limit, or a separate limit like we had with gpt 5.3 codex spark? i'm turning it off until the official rollout but get excited, the next few weeks are going to be amazing

here is the first look at realtime voice mode in codex the ui is already in the app, but the codex backend endpoint does not seem to be live yet i got it running by forcing the local feature gate and using my own openai api key, so it goes through the public realtime api instead it's incredible, but this demo alone cost me ~$4 gpt realtime 2 is very expensive, which makes me wonder how they're going to handle it - part of the normal usage limit, or a separate limit like we had with gpt 5.3 codex spark? i'm turning it off until the official rollout but get excited, the next few weeks are going to be amazing

🥔🥔🥔

39,508 views • 1 month ago

As we mark anotther #LesbianVisibilityWeek, this matters. You don’t have to be a lesbian to stand up for lesbians. You just have to believe in equality and fairness. Anyone can be an ally. Everyone has a role to play.

As we mark anotther #LesbianVisibilityWeek, this matters. You don’t have to be a lesbian to stand up for lesbians. You just have to believe in equality and fairness. Anyone can be an ally. Everyone has a role to play.

Dawn Butler ✊🏾💙

25,566 views • 3 months ago