Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

At OpenAI DevDay, we introduced the Realtime API with a demo of an interactive solar system you can navigate with your voice. A lot of you have asked how it was built, so we've open-sourced it—with a twist: this version uses our new WebRTC integration. Have fun exploring! 🌍

Katia Gil Guzman

6,644 subscribers

316,561 Aufrufe • vor 1 Jahr •via X (Twitter)

Bildung Wissenschaft & Technologie Nachrichten & Politik

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Katia Gil Guzman

Katia Gil Guzmanvor 1 Jahr

With the new WebRTC integration, you can connect to the Realtime API directly from the browser:

Profilbild von Katia Gil Guzman

Katia Gil Guzmanvor 1 Jahr

Shout out to @karoliskosas for the initial idea (I have to admit, it sounded a bit crazy when I first heard it) and to the @splinetool team for building an amazing tool and helping us optimize the scene!

Profilbild von Katia Gil Guzman

Katia Gil Guzmanvor 1 Jahr

@splinetool To see the full replay:

Profilbild von AssemblyAI

AssemblyAIvor 1 Jahr

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

Profilbild von Nicolas Camara

Nicolas Camaravor 1 Jahr

This is awesome! Thanks for open sourcing it!

Profilbild von ZAZO

ZAZOvor 1 Jahr

It’s super good

Profilbild von Drago Nicolas Bonacich

Drago Nicolas Bonacichvor 1 Jahr

This is super cool! Thanks!

Profilbild von oAI

oAIvor 1 Jahr

@OpenAIDevs that's what's missing in the AVM

Profilbild von Mbongeni Ndlovu

Mbongeni Ndlovuvor 1 Jahr

Appreciate it

Profilbild von John Schoenith

John Schoenithvor 1 Jahr

Look at this shiz! Open-source at OpenAI! 🪩

Profilbild von 🐧 lalo adrian morales 𝕏

🐧 lalo adrian morales 𝕏vor 1 Jahr

well that was easy ! thanks!

Ähnliche Videos

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵

Craig Dennis

60,122 Aufrufe • vor 1 Jahr

hey everyone! super excited play a small role in the OpenAI Realtime API launch today. we have launched an open source repo, the Realtime Console -- a React app you can hack apart and start building with immediately. here's a demo! thanks Peter Bakkum Mark Hudnall Jordan Sitkin

hey everyone! super excited play a small role in the OpenAI Realtime API launch today. we have launched an open source repo, the Realtime Console -- a React app you can hack apart and start building with immediately. here's a demo! thanks Peter Bakkum Mark Hudnall Jordan Sitkin

keith

243,232 Aufrufe • vor 1 Jahr

We just updated Junie and the Codex integration in JetBrains AI! Now you can use OpenAI GPT 5.5 in all JetBrains IDEs. We ran various tests with this model but have a look yourself at one of the demo projects we built (🔊 on)

We just updated Junie and the Codex integration in JetBrains AI! Now you can use OpenAI GPT 5.5 in all JetBrains IDEs. We ran various tests with this model but have a look yourself at one of the demo projects we built (🔊 on)

JetBrains

10,966 Aufrufe • vor 3 Monaten

We've been having a lot of fun with the newest Mega Kangaskhan ex card that was introduced in Fantastical Parade. What new card-inspired decks have you cooked up this set? Learn more:

We've been having a lot of fun with the newest Mega Kangaskhan ex card that was introduced in Fantastical Parade. What new card-inspired decks have you cooked up this set? Learn more:

Pokémon TCG Pocket

120,989 Aufrufe • vor 5 Monaten

First weekend building with Fable (it also helped put together this video) Open-sourced RC car raycast controller built with three.js + cannon. It includes mobile controls + Gamepad API support. It's also been a lot of fun implementing it with 3D Gaussian Splatting. Test it out!

First weekend building with Fable (it also helped put together this video) Open-sourced RC car raycast controller built with three.js + cannon. It includes mobile controls + Gamepad API support. It's also been a lot of fun implementing it with 3D Gaussian Splatting. Test it out!

Ian Curtis

30,043 Aufrufe • vor 1 Monat

That’s it. That’s the tweet. The Realtime API now supports WebRTC—you can add Realtime capabilities with just a handful of lines of code. We’ve also cut prices by 60%, added GPT-4o mini (10x cheaper than previous prices), improved voice quality, and made inputs more reliable.

That’s it. That’s the tweet. The Realtime API now supports WebRTC—you can add Realtime capabilities with just a handful of lines of code. We’ve also cut prices by 60%, added GPT-4o mini (10x cheaper than previous prices), improved voice quality, and made inputs more reliable.

OpenAI Developers

523,494 Aufrufe • vor 1 Jahr

Built a "YouTube realtime copilot" browser extension using OpenAI's realtime 2 API: The agent watches the video alongside you, and can answer any question you have about what was just said via realtime voice chat. The crazy part to me is: It can differentiate the YouTube's audio stream and your voice, so it doesn't confuse the video as commands, and stays silent unless you ask something!

Built a "YouTube realtime copilot" browser extension using OpenAI's realtime 2 API: The agent watches the video alongside you, and can answer any question you have about what was just said via realtime voice chat. The crazy part to me is: It can differentiate the YouTube's audio stream and your voice, so it doesn't confuse the video as commands, and stays silent unless you ask something!

Zara Zhang

45,125 Aufrufe • vor 3 Monaten

Sierra agents can be published on your web site, integrated with your mobile app, answer the phone - and now they can also be published to ChatGPT so you can directly reach hundreds of millions of consumers with your agent. Since OpenAI announced ChatGPT apps two weeks ago, we've heard from our customers that they want two things: a direct customer relationship and the ability to reach customers where they already are. We don't think you should have to choose. With Sierra, you can build once and run everywhere. Here's a demo of an agent I built to show how it works. And our blog post is here:

Sierra agents can be published on your web site, integrated with your mobile app, answer the phone - and now they can also be published to ChatGPT so you can directly reach hundreds of millions of consumers with your agent. Since OpenAI announced ChatGPT apps two weeks ago, we've heard from our customers that they want two things: a direct customer relationship and the ability to reach customers where they already are. We don't think you should have to choose. With Sierra, you can build once and run everywhere. Here's a demo of an agent I built to show how it works. And our blog post is here:

Bret Taylor

55,607 Aufrufe • vor 9 Monaten

Say hello to “X to Voice”. An open-source app that generates a unique voice from your X/Twitter profile. Built with our Voice Design API which is live from today — where you can create a unique voice from any prompt. Try it with your profile here:

ElevenLabs

833,680 Aufrufe • vor 1 Jahr

Who knew that inviting AI into your video chats could be fun && functional? We used Cloudflare Developers Calls and the OpenAI Realtime API with WebRTC to add AI to our open source Orange Meets demo app. I forked it to make my own murder mystery app 😈 Code and details in 🧵

Who knew that inviting AI into your video chats could be fun && functional? We used Cloudflare Developers Calls and the OpenAI Realtime API with WebRTC to add AI to our open source Orange Meets demo app. I forked it to make my own murder mystery app 😈 Code and details in 🧵

Ricky

16,796 Aufrufe • vor 1 Jahr

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

Matthew Lam

17,674 Aufrufe • vor 2 Monaten

You can fold a moving bat out of just one sheet of paper! Have fun with it with your little one!

You can fold a moving bat out of just one sheet of paper! Have fun with it with your little one!

Craft Gallery

36,853 Aufrufe • vor 5 Monaten

🐰 i worked a lot with El Guincho. I got to listen to a lot of tracks and i was really wowed by just how fresh and new this was. One of the first thoughts i had was ‘wow, you can approach a song like this?’ and i felt a lot of pressure about how to make the best use of it. so that must be why i wanted to stay close, it was a totally a different vibe, a whole new vibe. i hope to work again and have another session with the producers and songwriters that we worked with

🐰 i worked a lot with El Guincho. I got to listen to a lot of tracks and i was really wowed by just how fresh and new this was. One of the first thoughts i had was ‘wow, you can approach a song like this?’ and i felt a lot of pressure about how to make the best use of it. so that must be why i wanted to stay close, it was a totally a different vibe, a whole new vibe. i hope to work again and have another session with the producers and songwriters that we worked with

jk.97

26,798 Aufrufe • vor 4 Monaten

Screaming Frog's OpenAI integration allows you to automate tons of basic SEO tasks. This step by step tutorial to shows you how to set up ChatGPT crawling. As I posted yesterday, Screaming Frog last update was one of the biggest we've seen in a long time. They've introduced a suite of new features including the ability to upload custom JavaScript and having Screaming Frog execute it. This includes a direct integration with OpenAI. To set up OpenAI crawling, here are the steps to follow:

Screaming Frog's OpenAI integration allows you to automate tons of basic SEO tasks. This step by step tutorial to shows you how to set up ChatGPT crawling. As I posted yesterday, Screaming Frog last update was one of the biggest we've seen in a long time. They've introduced a suite of new features including the ability to upload custom JavaScript and having Screaming Frog execute it. This includes a direct integration with OpenAI. To set up OpenAI crawling, here are the steps to follow:

Chris Long

30,710 Aufrufe • vor 2 Jahren

Look at this. We've made it so you can see where market makers are directly positioned with our Periscope tool. It is a must have toolset for you. See more:

Look at this. We've made it so you can see where market makers are directly positioned with our Periscope tool. It is a must have toolset for you. See more:

unusual_whales

150,471 Aufrufe • vor 1 Tag

We've partnered with OpenAI as one of their early research preview partners for the new computer use model. With this model, AI can control a browser better than ever before 👀 Of course, we had to launch a free hosted version + open source implementation. Check it out!

We've partnered with OpenAI as one of their early research preview partners for the new computer use model. With this model, AI can control a browser better than ever before 👀 Of course, we had to launch a free hosted version + open source implementation. Check it out!

Browserbase

40,232 Aufrufe • vor 1 Jahr

Trump: We could have a lot of fun with these guys. You can imagine when they win, how they celebrate. We don't want to talk about it. We don't want to talk about it.

Trump: We could have a lot of fun with these guys. You can imagine when they win, how they celebrate. We don't want to talk about it. We don't want to talk about it.

Acyn

30,346 Aufrufe • vor 5 Monaten

I extended the range of Ace's chromatic (46 notes instead of 24) and EQ'd it a bit so you guys can have fun with his voice (+ a demo of what he sounds like in a quick manual blast remix I made lol) link:

I extended the range of Ace's chromatic (46 notes instead of 24) and EQ'd it a bit so you guys can have fun with his voice (+ a demo of what he sounds like in a quick manual blast remix I made lol) link:

Kamex 🇩🇴 | BLM

12,556 Aufrufe • vor 2 Jahren