Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Built a "YouTube realtime copilot" browser extension using OpenAI's realtime 2 API: The agent watches the video alongside you, and can answer any question you have about what was just said via realtime voice chat. The crazy part to me is: It can differentiate the YouTube's audio stream and...

zarazhangrui's profile picture

Zara Zhang

69,674 subscribers

44,631 Aufrufe • vor 2 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Want to chat over your own documents using the new OpenAI Realtime API? You can do so now using Voice Chat PDF, built using LlamaIndex 🦙 and Next.js. The video below shows an example using a document about physical mailing standards. 📄
0:20

Want to chat over your own documents using the new OpenAI Realtime API? You can do so now using Voice Chat PDF, built using LlamaIndex 🦙 and Next.js. The video below shows an example using a document about physical mailing standards. 📄

Marcus Schiesser

159,606 Aufrufe • vor 1 Jahr

You can now edit SolveIt messages via realtime conversational voice, and have the diff edits optionally tracked.
1:21

You can now edit SolveIt messages via realtime conversational voice, and have the diff edits optionally tracked.

David Gwyer - AI & ML Researcher/Engineer

16,626 Aufrufe • vor 2 Monaten

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.
4:03

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.

OpenAI

3,648,946 Aufrufe • vor 2 Monaten

Next, the Traces dashboard now supports Realtime API sessions, letting you visualize voice agent runs, including audio input/output, tool invocations, and interruptions, whether created via the API or the Agents SDK. Here's Alex Gamble to show you how it works:
1:06

Next, the Traces dashboard now supports Realtime API sessions, letting you visualize voice agent runs, including audio input/output, tool invocations, and interruptions, whether created via the API or the Agents SDK. Here's Alex Gamble to show you how it works:

OpenAI Developers

13,411 Aufrufe • vor 1 Jahr

The gap between “Ethereum will be realtime” and “Ethereum is realtime” is what ETHGas is working on. When Ethereum is realtime, it changes the entire DeFi landscape and turns big claims into something you can actually measure.
0:49

The gap between “Ethereum will be realtime” and “Ethereum is realtime” is what ETHGas is working on. When Ethereum is realtime, it changes the entire DeFi landscape and turns big claims into something you can actually measure.

ETHGAS

42,099 Aufrufe • vor 5 Monaten

Base44 is now available as a standalone backend, designed for the way software is built today: CLI-first, realtime, and AI-agent friendly. You can now deploy a production-ready backend - auth, database, realtime, and hosting - directly from the CLI
0:39

Base44 is now available as a standalone backend, designed for the way software is built today: CLI-first, realtime, and AI-agent friendly. You can now deploy a production-ready backend - auth, database, realtime, and hosting - directly from the CLI

Maor Shlomo

14,611 Aufrufe • vor 5 Monaten

Voice is the OG modality. So excited for image inputs, function calling & MCP support in the Realtime API GA! `gpt-realtime` is a lot more natural and expressive, and every time a SOTA voice model is released, you know what I gotta do... Here is the new voice Marin, on KaraokeBench! scores 3 out of 7 again from me
10:41

Voice is the OG modality. So excited for image inputs, function calling & MCP support in the Realtime API GA! `gpt-realtime` is a lot more natural and expressive, and every time a SOTA voice model is released, you know what I gotta do... Here is the new voice Marin, on KaraokeBench! scores 3 out of 7 again from me

swyx

51,219 Aufrufe • vor 10 Monaten

Introducing Realtime TTS-2, a new generation of voice model built for realtime conversation. It is the first voice model that hears the conversation, takes natural-language voice direction, holds one voice identity across over 100 languages, and speaks like a person who is paying attention. The result is voice AI that feels as good as it sounds. Try it out: Learn More:
2:06

Introducing Realtime TTS-2, a new generation of voice model built for realtime conversation. It is the first voice model that hears the conversation, takes natural-language voice direction, holds one voice identity across over 100 languages, and speaks like a person who is paying attention. The result is voice AI that feels as good as it sounds. Try it out: Learn More:

Inworld AI

325,540 Aufrufe • vor 2 Monaten

So you can just give it a postgres URL (or sqlite db file) and it will just... explore it and offer to build things. Even with a crazy nonstandard (from Ecto perspective) database. The search bar crashed, I sent the logs to the agent, and boom realtime searching
1:27

So you can just give it a postgres URL (or sqlite db file) and it will just... explore it and offer to build things. Even with a crazy nonstandard (from Ecto perspective) database. The search bar crashed, I sent the logs to the agent, and boom realtime searching

Chris McCord

12,758 Aufrufe • vor 1 Jahr

That’s it. That’s the tweet. The Realtime API now supports WebRTC—you can add Realtime capabilities with just a handful of lines of code. We’ve also cut prices by 60%, added GPT-4o mini (10x cheaper than previous prices), improved voice quality, and made inputs more reliable.
5:36

That’s it. That’s the tweet. The Realtime API now supports WebRTC—you can add Realtime capabilities with just a handful of lines of code. We’ve also cut prices by 60%, added GPT-4o mini (10x cheaper than previous prices), improved voice quality, and made inputs more reliable.

OpenAI Developers

523,494 Aufrufe • vor 1 Jahr

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵
1:00

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵

Craig Dennis

60,091 Aufrufe • vor 1 Jahr

Talk to Gemma 4 31B with our voice app! It sees and searches the web faster than you blink. Thanks to Cerebras’ ultra fast inference, the LLM is almost instantaneous. The whole stack is fully open-source, and is a drop-in replacement for OpenAI's realtime API. Demo:
1:46

Talk to Gemma 4 31B with our voice app! It sees and searches the web faster than you blink. Thanks to Cerebras’ ultra fast inference, the LLM is almost instantaneous. The whole stack is fully open-source, and is a drop-in replacement for OpenAI's realtime API. Demo:

Andi Marafioti

88,125 Aufrufe • vor 15 Tagen

Today I built this voice agent while on a call with it Introducing /this-needs-a-call - it's a slash command to start a phone call - grill or be grilled by the voice agent in realtime - your coding agent listens in on the call via MCP and works while you talk for hours
6:15

Today I built this voice agent while on a call with it Introducing /this-needs-a-call - it's a slash command to start a phone call - grill or be grilled by the voice agent in realtime - your coding agent listens in on the call via MCP and works while you talk for hours

jacob paris ▲

32,463 Aufrufe • vor 16 Tagen

At OpenAI DevDay, we introduced the Realtime API with a demo of an interactive solar system you can navigate with your voice. A lot of you have asked how it was built, so we've open-sourced it—with a twist: this version uses our new WebRTC integration. Have fun exploring! 🌍
1:44

At OpenAI DevDay, we introduced the Realtime API with a demo of an interactive solar system you can navigate with your voice. A lot of you have asked how it was built, so we've open-sourced it—with a twist: this version uses our new WebRTC integration. Have fun exploring! 🌍

Katia Gil Guzman

316,561 Aufrufe • vor 1 Jahr

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)
3:13

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

54,912 Aufrufe • vor 2 Monaten

Huge News: Base44 is now available as a standalone backend, designed for the way software is built today: CLI-first, realtime, and AI-agent friendly. We’re opening up Base44’s backend as a standalone product. You can now deploy a production-ready backend - auth, database, realtime, and hosting - directly from the CLI.
0:50

Huge News: Base44 is now available as a standalone backend, designed for the way software is built today: CLI-first, realtime, and AI-agent friendly. We’re opening up Base44’s backend as a standalone product. You can now deploy a production-ready backend - auth, database, realtime, and hosting - directly from the CLI.

Base44

314,964 Aufrufe • vor 5 Monaten

Introducing Genspark Realtime Voice. I connected Genspark to my car and headed to work. With Genspark Realtime Voice, prepping for the day feels as easy as chatting. It can check my schedule, send emails and messages, look things up, and even make me a commute playlist. It can also help me get real work done — generate slides, run deep research, analyze data, and more. Just talk. Fully hands-free!
2:16

Introducing Genspark Realtime Voice. I connected Genspark to my car and headed to work. With Genspark Realtime Voice, prepping for the day feels as easy as chatting. It can check my schedule, send emails and messages, look things up, and even make me a commute playlist. It can also help me get real work done — generate slides, run deep research, analyze data, and more. Just talk. Fully hands-free!

Genspark

169,275 Aufrufe • vor 3 Monaten

What if you never had to answer the same question twice? Delphi, you can now create a digital version of you - just by being interviewed. It learns how you think and what you know, so it can answer questions and connect with others 24/7, in your voice
1:08

What if you never had to answer the same question twice? Delphi, you can now create a digital version of you - just by being interviewed. It learns how you think and what you know, so it can answer questions and connect with others 24/7, in your voice

Dara

42,416 Aufrufe • vor 8 Monaten