Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🚀 ChatGPT Realtime API in ACTION 🌐 Here's the application Integrated with: 🌦️ Realtime Weather 🧠 Ability to Save in Memory 🔧 Uses Function Calling Use this code and extend it from here! 💡👨‍💻

Mervin Praison

11,350 subscribers

42,982 views • 1 year ago •via X (Twitter)

News & Politics Science & Technology

Anya Rossi• Live Now

Private livecam show

7 Comments

Shawn1 year ago

The price is gonna have to come way down and this will be really popular

🐧 lalo adrian morales 𝕏1 year ago

heckkkkk yeah!! Its gettin COOOOLER

steve ike1 year ago

Super cool demo. Nice work! Looking forward to playing with the realtime APIs soon.

Dean1 year ago

This is brilliant, @MervinPraison. How was the latency between your speech and the response?

Mervin Praison1 year ago

It was immediate. Didn't notice any lag.

ZAZO1 year ago

Memory function is killer

Shajeel Afzal1 year ago

Is it builtin memory or did you do it yourself?

Related Videos

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵

I've been having a ton of fun exploring the Realtime API from OpenAI. Today they just launched the ability to connect to it over WebRTC! 🤖🖐️ It supports Tool Calling, so check out what that looks like in Realtime function calling in any browser based app. I built a template that allows for relaying through a Cloudflare Developers Worker so you can do it too! Full 📼 in 🧵

Craig Dennis

60,122 views • 1 year ago

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

Matthew Lam

17,674 views • 2 months ago

Voice is the OG modality. So excited for image inputs, function calling & MCP support in the Realtime API GA! `gpt-realtime` is a lot more natural and expressive, and every time a SOTA voice model is released, you know what I gotta do... Here is the new voice Marin, on KaraokeBench! scores 3 out of 7 again from me

Voice is the OG modality. So excited for image inputs, function calling & MCP support in the Realtime API GA! `gpt-realtime` is a lot more natural and expressive, and every time a SOTA voice model is released, you know what I gotta do... Here is the new voice Marin, on KaraokeBench! scores 3 out of 7 again from me

swyx

51,219 views • 11 months ago

Nano Banana Lite is fast! 🍌🚀 I put it to the test with the Interactions API and Gemini Live to illustrate stories in near-realtime 🤯👇 Try it out now 👉

Nano Banana Lite is fast! 🍌🚀 I put it to the test with the Interactions API and Gemini Live to illustrate stories in near-realtime 🤯👇 Try it out now 👉

Thor 雷神 ⚡️

13,287 views • 1 month ago

another realtime api trick i stumbled upon 💡 to prevent the model from "jumping in" to speak whenever you pause, you can give it a stay_silent() function that does nothing it'll call it when it knows you haven't finished your thought, even if you pause - full prompt below👇 -

another realtime api trick i stumbled upon 💡 to prevent the model from "jumping in" to speak whenever you pause, you can give it a stay_silent() function that does nothing it'll call it when it knows you haven't finished your thought, even if you pause - full prompt below👇 -

ilan bigio

74,247 views • 1 year ago

Here's the crazy #smilingfriends Jesse puppet in action. Had such a fun time making it and working with ZAM STUDIOS / Aron Fromm was a breeze! #UnrealEngine #RealTime #Zbrush #Houdini

Here's the crazy #smilingfriends Jesse puppet in action. Had such a fun time making it and working with ZAM STUDIOS / Aron Fromm was a breeze! #UnrealEngine #RealTime #Zbrush #Houdini

Ryan Corniel

1,482,382 views • 9 months ago

Test the real-time GI function in #Unity6, without any baking, and only use real-time GI to achieve the sense of volume.😊 Come and subscribe to my Fanbox to download this project #Unity #VFX #shader #realtime

Test the real-time GI function in #Unity6, without any baking, and only use real-time GI to achieve the sense of volume.😊 Come and subscribe to my Fanbox to download this project #Unity #VFX #shader #realtime

Sakura Rabbit 樱花兔

66,795 views • 2 years ago

The week in OpenAI and Anthropic news (Week 19, 2026) OpenAI rolled out GPT-5.5 Instant as the new ChatGPT default model with memory sources and ChatGPT for Excel and Google Sheets globally, launched three new realtime voice models in the API (GPT-Realtime-2, GPT-Realtime-Translate, GPT-Realtime-Whisper) and an OpenAI CLI, introduced Trusted Contact safety feature, GPT-5.5-Cyber for defenders, B2B Signals report, ChatGPT Futures Class of 2026, EMEA youth safety blueprint, privacy in model training explainer, expanded ads pilot, published engineering posts on low-latency voice, MRC supercomputer networking, running Codex safely internally, and investigating accidental chain-of-thought grading during reinforcement learning, plus discovered ChatGPT Personal Wiki and dropped a goblin-themed merch line that sold out Anthropic hosted Code with Claude developer conference in San Francisco, announced a new enterprise AI services company with Blackstone, Hellman & Friedman, and Goldman Sachs, signed a SpaceX compute partnership and raised Claude Code and API usage limits, made Claude for Excel, PowerPoint, and Word generally available with Claude for Outlook in beta, launched Workload Identity Federation, financial services agent templates, dreaming, outcomes, and multiagent orchestration in Managed Agents, shipped 60+ Claude Code reliability fixes, published research on agentic misalignment training, sandbagging mitigation, model spec midtraining, and Natural Language Autoencoders, donated Petri to Meridian Labs, introduced The Anthropic Institute research agenda, plus discovered Orbit proactive assistant for Cowork and /radio command in Claude Code, and more

The week in OpenAI and Anthropic news (Week 19, 2026) OpenAI rolled out GPT-5.5 Instant as the new ChatGPT default model with memory sources and ChatGPT for Excel and Google Sheets globally, launched three new realtime voice models in the API (GPT-Realtime-2, GPT-Realtime-Translate, GPT-Realtime-Whisper) and an OpenAI CLI, introduced Trusted Contact safety feature, GPT-5.5-Cyber for defenders, B2B Signals report, ChatGPT Futures Class of 2026, EMEA youth safety blueprint, privacy in model training explainer, expanded ads pilot, published engineering posts on low-latency voice, MRC supercomputer networking, running Codex safely internally, and investigating accidental chain-of-thought grading during reinforcement learning, plus discovered ChatGPT Personal Wiki and dropped a goblin-themed merch line that sold out Anthropic hosted Code with Claude developer conference in San Francisco, announced a new enterprise AI services company with Blackstone, Hellman & Friedman, and Goldman Sachs, signed a SpaceX compute partnership and raised Claude Code and API usage limits, made Claude for Excel, PowerPoint, and Word generally available with Claude for Outlook in beta, launched Workload Identity Federation, financial services agent templates, dreaming, outcomes, and multiagent orchestration in Managed Agents, shipped 60+ Claude Code reliability fixes, published research on agentic misalignment training, sandbagging mitigation, model spec midtraining, and Natural Language Autoencoders, donated Petri to Meridian Labs, introduced The Anthropic Institute research agenda, plus discovered Orbit proactive assistant for Cowork and /radio command in Claude Code, and more

Tibor Blaho

12,602 views • 3 months ago

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

55,222 views • 3 months ago

We've been shipping a lot with our Visual Studio Code weekly releases! 🚀 Here's a roundup of some of the highlights from the past month: 🤖 Autopilot 🌐 Integrated browser debugging 🎨 Refreshed themes 📸 Images & video in chat … and much more!

We've been shipping a lot with our Visual Studio Code weekly releases! 🚀 Here's a roundup of some of the highlights from the past month: 🤖 Autopilot 🌐 Integrated browser debugging 🎨 Refreshed themes 📸 Images & video in chat … and much more!

Visual Studio Code

32,412 views • 4 months ago

Watch this demo from Thor 雷神 ⚡️ to see Gemini 3.1 Flash Live in action as a real-time DJ. The model uses function calling (to the Gemini API) to generate custom 30-second clips using Lyria 3️⃣ Start your own studio session in Google AI Studio:

Watch this demo from Thor 雷神 ⚡️ to see Gemini 3.1 Flash Live in action as a real-time DJ. The model uses function calling (to the Gemini API) to generate custom 30-second clips using Lyria 3️⃣ Start your own studio session in Google AI Studio:

Google AI Developers

11,216 views • 3 months ago

This demo is insane. A student shares their iPad screen with the new ChatGPT + GPT-4o, and the AI speaks with them and helps them learn in *realtime*. Imagine giving this to every student in the world. The future is so, so bright.

This demo is insane. A student shares their iPad screen with the new ChatGPT + GPT-4o, and the AI speaks with them and helps them learn in realtime. Imagine giving this to every student in the world. The future is so, so bright.

Mckay Wrigley

10,930,224 views • 2 years ago

Today, we’re excited to launch Razorpay Blade's MCP! 🚀 🎨 Blade is a customizable, themeable design system. Use it as is, or bring your brand colors. 💡 With MCP, go from prompt to code in seconds, with the quality of a battle tested DS. 🌐 OSS →

Razorpay Tech

13,693 views • 1 year ago

I've been working with Jakub Červený to integrate our #GaussianSplatting renderer with #threejs, I created this infinite portal demo as a feature showcase! You can play with it live here This is captured in realtime on my mac, here's how it works: 🧵

I've been working with Jakub Červený to integrate our #GaussianSplatting renderer with #threejs, I created this infinite portal demo as a feature showcase! You can play with it live here This is captured in realtime on my mac, here's how it works: 🧵

🌈 George Corney

101,648 views • 2 years ago

Fully realtime 3D inflation shader in Figma. Pretty hyped about this one! Need to polish it up and make the UI a bit easier to use but will share in a couple of days. Soooo much fun to play with!

Fully realtime 3D inflation shader in Figma. Pretty hyped about this one! Need to polish it up and make the UI a bit easier to use but will share in a couple of days. Soooo much fun to play with!

Piers Cowburn

16,586 views • 1 month ago

Function calling is a powerful way to extend the capabilities of LLMs and AI agents by letting them use external tools. Our new short course Function calling and Data Extraction with LLMs, created with @NexusflowX and taught by Jiantao Jiao and Venkat, demonstrates how to prompt LLMs to form calls to external functions. You'll work with NexusRavenV2-13B, a 13B parameter open-source model that excels in function calling tasks while still being small enough to host locally. Learn to use function calling to extract structured data from unstructured text and access web APIs, and build an end-to-end application that processes customer service transcripts. You'll learn how to build LLM-powered applications that can analyze feedback, automate data entry, and enhance search. Please get started here:

Function calling is a powerful way to extend the capabilities of LLMs and AI agents by letting them use external tools. Our new short course Function calling and Data Extraction with LLMs, created with @NexusflowX and taught by Jiantao Jiao and Venkat, demonstrates how to prompt LLMs to form calls to external functions. You'll work with NexusRavenV2-13B, a 13B parameter open-source model that excels in function calling tasks while still being small enough to host locally. Learn to use function calling to extract structured data from unstructured text and access web APIs, and build an end-to-end application that processes customer service transcripts. You'll learn how to build LLM-powered applications that can analyze feedback, automate data entry, and enhance search. Please get started here:

Andrew Ng

110,606 views • 2 years ago

here is the first look at realtime voice mode in codex the ui is already in the app, but the codex backend endpoint does not seem to be live yet i got it running by forcing the local feature gate and using my own openai api key, so it goes through the public realtime api instead it's incredible, but this demo alone cost me ~$4 gpt realtime 2 is very expensive, which makes me wonder how they're going to handle it - part of the normal usage limit, or a separate limit like we had with gpt 5.3 codex spark? i'm turning it off until the official rollout but get excited, the next few weeks are going to be amazing

here is the first look at realtime voice mode in codex the ui is already in the app, but the codex backend endpoint does not seem to be live yet i got it running by forcing the local feature gate and using my own openai api key, so it goes through the public realtime api instead it's incredible, but this demo alone cost me ~$4 gpt realtime 2 is very expensive, which makes me wonder how they're going to handle it - part of the normal usage limit, or a separate limit like we had with gpt 5.3 codex spark? i'm turning it off until the official rollout but get excited, the next few weeks are going to be amazing

🥔🥔🥔

39,508 views • 2 months ago

It’s like Bloomberg meets the world monitor Elie Habib I put Virat Singh financial dataset api to cross reference all the realtime data to make realtime financial decisions and executions. Got multiple agents already inside Convex style and even got Grok with xAI feed This is one of the most powerful GitHub’s I have ever forked in my life brother

It’s like Bloomberg meets the world monitor Elie Habib I put Virat Singh financial dataset api to cross reference all the realtime data to make realtime financial decisions and executions. Got multiple agents already inside Convex style and even got Grok with xAI feed This is one of the most powerful GitHub’s I have ever forked in my life brother

8Bit🦞

20,161 views • 5 months ago