Загрузка видео...

Не удалось загрузить видео

На главную

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a...

craigsdennis's profile picture

Craig Dennis

7,414 subscribers

60,122 просмотров • 1 год назад •via X (Twitter)

ОбразованиеИгрыНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Craig Dennis
Craig Dennis1 год назад

Full 📼

Фото профиля mobilewatcher2
mobilewatcher21 год назад

Cloudless remote access to the app | With this app you are in control. Pay for the app and get Detect Events and Smoke Alarm, Record Video. Text and email will increase awareness. Access remotely using QR code on MobileWatcher2 ™ Main screen top.

Фото профиля Josh Larson
Josh Larson1 год назад

@OpenAI I went from "wow this is neat" to "gosh I should build something with this!" to "I MUST buy a robot hand"

Фото профиля Craig Dennis
Craig Dennis1 год назад

@OpenAI Put it on your holiday list! Link in the repo!

Фото профиля Alex Volkov (Thursd/AI)
Alex Volkov (Thursd/AI)1 год назад

@OpenAI This is SO dope! 🔥

Фото профиля Brian Ball
Brian Ball1 год назад

@OpenAI "It's kinda hard to read.. can you fix that?"

Фото профиля Craig Dennis
Craig Dennis1 год назад

@OpenAI Bonkers right?!!!

Фото профиля William Allen
William Allen1 год назад

@OpenAI 🤖🤯!!

Фото профиля Ray Fernando
Ray Fernando1 год назад

@OpenAI Wow!! This is so cool!!

Фото профиля jensen
jensen1 год назад

@OpenAI This is great. Made a relay using DO a few months ago, but WebRTC makes way more sense.

Фото профиля Ryan Morrison 🇺🇸 🇿🇦
Ryan Morrison 🇺🇸 🇿🇦1 год назад

@OpenAI Very cool

Похожие видео

Who knew that inviting AI into your video chats could be fun && functional? We used Cloudflare Developers Calls and the OpenAI Realtime API with WebRTC to add AI to our open source Orange Meets demo app. I forked it to make my own murder mystery app 😈 Code and details in 🧵
4:17

Who knew that inviting AI into your video chats could be fun && functional? We used Cloudflare Developers Calls and the OpenAI Realtime API with WebRTC to add AI to our open source Orange Meets demo app. I forked it to make my own murder mystery app 😈 Code and details in 🧵

Ricky

16,796 просмотров • 1 год назад

At OpenAI DevDay, we introduced the Realtime API with a demo of an interactive solar system you can navigate with your voice. A lot of you have asked how it was built, so we've open-sourced it—with a twist: this version uses our new WebRTC integration. Have fun exploring! 🌍
1:44

At OpenAI DevDay, we introduced the Realtime API with a demo of an interactive solar system you can navigate with your voice. A lot of you have asked how it was built, so we've open-sourced it—with a twist: this version uses our new WebRTC integration. Have fun exploring! 🌍

Katia Gil Guzman

316,561 просмотров • 1 год назад

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!
0:28

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

Matthew Lam

17,674 просмотров • 2 месяцев назад

Built a "YouTube realtime copilot" browser extension using OpenAI's realtime 2 API: The agent watches the video alongside you, and can answer any question you have about what was just said via realtime voice chat. The crazy part to me is: It can differentiate the YouTube's audio stream and your voice, so it doesn't confuse the video as commands, and stays silent unless you ask something!
1:18

Built a "YouTube realtime copilot" browser extension using OpenAI's realtime 2 API: The agent watches the video alongside you, and can answer any question you have about what was just said via realtime voice chat. The crazy part to me is: It can differentiate the YouTube's audio stream and your voice, so it doesn't confuse the video as commands, and stays silent unless you ask something!

Zara Zhang

45,125 просмотров • 2 месяцев назад

Wow, Anthropic Claude Code is AWESOME I built a Cloudflare Developers Workers AI-powered app in just a few minutes, letting Claude do 99% of the work Check it out ⬇️ 🧵with a few other resources too
8:41

Wow, Anthropic Claude Code is AWESOME I built a Cloudflare Developers Workers AI-powered app in just a few minutes, letting Claude do 99% of the work Check it out ⬇️ 🧵with a few other resources too

Kristian

34,661 просмотров • 1 год назад

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)
3:13

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

55,222 просмотров • 3 месяцев назад

Voice is the OG modality. So excited for image inputs, function calling & MCP support in the Realtime API GA! `gpt-realtime` is a lot more natural and expressive, and every time a SOTA voice model is released, you know what I gotta do... Here is the new voice Marin, on KaraokeBench! scores 3 out of 7 again from me
10:41

Voice is the OG modality. So excited for image inputs, function calling & MCP support in the Realtime API GA! `gpt-realtime` is a lot more natural and expressive, and every time a SOTA voice model is released, you know what I gotta do... Here is the new voice Marin, on KaraokeBench! scores 3 out of 7 again from me

swyx

51,219 просмотров • 11 месяцев назад

another realtime api trick i stumbled upon 💡 to prevent the model from "jumping in" to speak whenever you pause, you can give it a stay_silent() function that does nothing it'll call it when it knows you haven't finished your thought, even if you pause - full prompt below👇 -
1:01

another realtime api trick i stumbled upon 💡 to prevent the model from "jumping in" to speak whenever you pause, you can give it a stay_silent() function that does nothing it'll call it when it knows you haven't finished your thought, even if you pause - full prompt below👇 -

ilan bigio

74,247 просмотров • 1 год назад

I love calling out these accounts from overseas that have zero clue what it takes to purchase a firearm here in the United States. In fact, purchasing a firearm is one of the most strict things that you can do here. So let’s make it as easy to purchase a firearm as it is to vote.
0:38

I love calling out these accounts from overseas that have zero clue what it takes to purchase a firearm here in the United States. In fact, purchasing a firearm is one of the most strict things that you can do here. So let’s make it as easy to purchase a firearm as it is to vote.

Steve 🇺🇸

23,181 просмотров • 1 месяц назад

Last week we released OpenAI Developers gpt-realtime our latest speech-to-speech model. Prompting S2S models can be extremely powerful. To get the most out of the model we compiled a series of prompting tips of what we saw work with early customers. To present them here's Cedar our new voice in gpt-realtime walk you through it. If you want to read it yourself check the links below 👇
2:08

Last week we released OpenAI Developers gpt-realtime our latest speech-to-speech model. Prompting S2S models can be extremely powerful. To get the most out of the model we compiled a series of prompting tips of what we saw work with early customers. To present them here's Cedar our new voice in gpt-realtime walk you through it. If you want to read it yourself check the links below 👇

dominik kundel

66,904 просмотров • 11 месяцев назад

Voice is an interface, and I feel we’re going to see voice explode. OpenAI gpt-realtime is out and allows for native speech-to-speech and precise tool calling in real time. It can also take image inputs and place SIP phone calls - good for a quick triage before human handoff.
1:20

Voice is an interface, and I feel we’re going to see voice explode. OpenAI gpt-realtime is out and allows for native speech-to-speech and precise tool calling in real time. It can also take image inputs and place SIP phone calls - good for a quick triage before human handoff.

Allie K. Miller

80,345 просмотров • 11 месяцев назад

OPENAI 🔥: The built-in web browser in the ChatGPT app is becoming more mature. Now it supports URL suggestions during typing. Besides that, the ChatGPT Chrome extension can now reference open tabs, highlight to ask, and more! I may need to try it as a default 👀
0:55

OPENAI 🔥: The built-in web browser in the ChatGPT app is becoming more mature. Now it supports URL suggestions during typing. Besides that, the ChatGPT Chrome extension can now reference open tabs, highlight to ask, and more! I may need to try it as a default 👀

🚨 AI News | TestingCatalog

62,691 просмотров • 5 дней назад

we've been disappointed at how the input into most apps is still just mouse and keyboard. So we built a tool that lets you write directly into any app - check it out
0:36

we've been disappointed at how the input into most apps is still just mouse and keyboard. So we built a tool that lets you write directly into any app - check it out

WillScuderi

3,741,838 просмотров • 2 лет назад

i hate the way tool calling works today, so i built a fix for it its called `executor` and it lets your agent run code to call any API / MCP / GraphQL server rather than installing CLIs this is a soft launch, you can try it with npm i -g executor but im still making it great
1:19

i hate the way tool calling works today, so i built a fix for it its called `executor` and it lets your agent run code to call any API / MCP / GraphQL server rather than installing CLIs this is a soft launch, you can try it with npm i -g executor but im still making it great

Rhys

129,065 просмотров • 5 месяцев назад

you don’t need a robot to see physical ai in action, you just need a browser. this week I finally tried something I've wanted to do for a while: design an SO-101 scene in three.js, connect it to a real leader arm, collect teleoperation data, train an ACT model, then port it to run in the browser. the result is the first ever demo of ACT that runs locally on the web. try it yourself now at
0:35

you don’t need a robot to see physical ai in action, you just need a browser. this week I finally tried something I've wanted to do for a while: design an SO-101 scene in three.js, connect it to a real leader arm, collect teleoperation data, train an ACT model, then port it to run in the browser. the result is the first ever demo of ACT that runs locally on the web. try it yourself now at

Binh Pham

120,916 просмотров • 1 месяц назад

we've been disappointed at how the input into most apps is still just mouse and keyboard. So we built a tool that lets you write directly into any app - check it out
0:23

we've been disappointed at how the input into most apps is still just mouse and keyboard. So we built a tool that lets you write directly into any app - check it out

WillScuderi

5,826,968 просмотров • 2 лет назад

we've been disappointed at how the input into most apps is still just mouse and keyboard. So we built a tool that lets you write directly into any app - check it out
0:30

we've been disappointed at how the input into most apps is still just mouse and keyboard. So we built a tool that lets you write directly into any app - check it out

WillScuderi

3,069,494 просмотров • 2 лет назад