Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

here is the first look at realtime voice mode in codex the ui is already in the app, but the codex backend endpoint does not seem to be live yet i got it running by forcing the local feature gate and using my own openai api key, so it... show more

🥔🥔🥔

4,436 subscribers

39,508 views • 3 months ago •via X (Twitter)

Gaming News & Politics Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

Matthew Lam

17,674 views • 3 months ago

GPT Image 2 + Codex: or how to make Codex not suck at UI. Step 1: Generate a UI image (native in Codex) Step 2: Get Codex to implement the UI based on it Step 3: Get Codex to iterate until it aligns with the image as much as possible Codex is bad at initial UI, but very good at implementing a reference design, so this is your way out - iterate with the image model first and then Codex will do a good job.

GPT Image 2 + Codex: or how to make Codex not suck at UI. Step 1: Generate a UI image (native in Codex) Step 2: Get Codex to implement the UI based on it Step 3: Get Codex to iterate until it aligns with the image as much as possible Codex is bad at initial UI, but very good at implementing a reference design, so this is your way out - iterate with the image model first and then Codex will do a good job.

Peter Gostev

67,245 views • 4 months ago

I have been using GPT-5.3-Codex-Spark all day every day for 2 weeks now. It's incredibly fast (Cerebras) I had a few moments where I felt I could see the future. So far I've tried it in: - Zed - Openclaw - Codex app - Codex cli / pi This is a clue to where we're heading.

I have been using GPT-5.3-Codex-Spark all day every day for 2 weeks now. It's incredibly fast (Cerebras) I had a few moments where I felt I could see the future. So far I've tried it in: - Zed - Openclaw - Codex app - Codex cli / pi This is a clue to where we're heading.

0xSero

22,212 views • 6 months ago

People are wiring Kimi K3 straight into Codex to skip its own app The model is elite. The Kimi app is the weak part. So builders connect their Kimi subscription to Codex through OAuth instead 👇 ▫️ K3 shows up in the model picker next to Sol 5.6 ▫️ No API key, no extra subscription ▫️ Sign in with OAuth, done The frontier model, running inside the coding tool people actually like using. When the model outgrows the app, users just route around it

People are wiring Kimi K3 straight into Codex to skip its own app The model is elite. The Kimi app is the weak part. So builders connect their Kimi subscription to Codex through OAuth instead 👇 ▫️ K3 shows up in the model picker next to Sol 5.6 ▫️ No API key, no extra subscription ▫️ Sign in with OAuth, done The frontier model, running inside the coding tool people actually like using. When the model outgrows the app, users just route around it

Ruben

58,369 views • 1 month ago

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

55,222 views • 4 months ago

ChatGPT Voice is now in the desktop app. Control your computer and direct multiple agents running in ChatGPT Work or Codex, using just your voice. It's powered by GPT-Live, so it can speak, listen, and coordinate work in the app at the same time. Rolling out globally today on macOS and Windows to Plus, Pro, Business, Edu, and Enterprise plans.

ChatGPT Voice is now in the desktop app. Control your computer and direct multiple agents running in ChatGPT Work or Codex, using just your voice. It's powered by GPT-Live, so it can speak, listen, and coordinate work in the app at the same time. Rolling out globally today on macOS and Windows to Plus, Pro, Business, Edu, and Enterprise plans.

OpenAI

4,728,540 views • 1 month ago

Command Code is now in Codex!! The plan we're on is $1 a month. It comes with $10 of credits and around 15K requests. GPT-5.6 Luna is in it, the same model we just moved to when the OpenCode sub got cut, and Grok 4.5 sits in there too next to DeepSeek, Kimi, GLM and MiniMax. Their own docs say the $1 tier has no API access. It's in our picker anyway. So there's a dollar sub in there next to everything else now. Next time a limit hits mid loop we grab it and keep going.

Command Code is now in Codex!! The plan we're on is $1 a month. It comes with $10 of credits and around 15K requests. GPT-5.6 Luna is in it, the same model we just moved to when the OpenCode sub got cut, and Grok 4.5 sits in there too next to DeepSeek, Kimi, GLM and MiniMax. Their own docs say the $1 tier has no API access. It's in our picker anyway. So there's a dollar sub in there next to everything else now. Next time a limit hits mid loop we grab it and keep going.

Ziwen

120,308 views • 1 month ago

WOW! I'm so excited about this. OpenAI Developers said Codex was good at Computer Use, but I wasn't prepared for this. For the last two weeks I've been working on a Computer Use skill to work with Linux. And while I had some success, it was a pretty frustrating experience. That is...until the breakthrough. Using accessibility tools, Codex can now control my entire computer, not just the browser. There are limits to this, of course, but what a time to be alive. This Computer Use skill will unlock and entirely new set of automations, all powered by Codex. Demonstration below 👇

WOW! I'm so excited about this. OpenAI Developers said Codex was good at Computer Use, but I wasn't prepared for this. For the last two weeks I've been working on a Computer Use skill to work with Linux. And while I had some success, it was a pretty frustrating experience. That is...until the breakthrough. Using accessibility tools, Codex can now control my entire computer, not just the browser. There are limits to this, of course, but what a time to be alive. This Computer Use skill will unlock and entirely new set of automations, all powered by Codex. Demonstration below 👇

am.will

79,698 views • 5 months ago

yesterday, at the Codex hackathon, i ended up solving this problem by building an MCP connector app that turns ChatGPT into Codex, sort of. 1. you get to use GPT-5.5 Pro by exposing your local dev environment to ChatGPT, so you can use 5.5 pro for planning and handover to codex to execute it 2. you get to 2x your rate limits, just like last month, because apparently ChatGPT and Codex have separate rate limits here is the quick demo

yesterday, at the Codex hackathon, i ended up solving this problem by building an MCP connector app that turns ChatGPT into Codex, sort of. 1. you get to use GPT-5.5 Pro by exposing your local dev environment to ChatGPT, so you can use 5.5 pro for planning and handover to codex to execute it 2. you get to 2x your rate limits, just like last month, because apparently ChatGPT and Codex have separate rate limits here is the quick demo

waishnav

556,306 views • 3 months ago

I just built an entire app in 7 minutes using Codex Spark. EveryMCP: a universal MCP server installer that configures 15+ coding agents from a single CLI command. 12 Codex Spark subagents. All running in parallel. At the same time. The project itself isn't complicated, but just imagine what this looks like when you scale the complexity. My setup: • Codex 5.3 High to plan • Codex 5.3 Medium to orchestrate • 12 Codex Spark agents (xhigh) to implement in one shot Spark isn't competing with the big models. It's a different tool entirely. You orchestrate it. You run it in swarms. And it moves at a speed that wasn't possible before, even for the top 1% of AI coders. More complex workflows coming soon. Comment below if you want me to tell you how I did it.

I just built an entire app in 7 minutes using Codex Spark. EveryMCP: a universal MCP server installer that configures 15+ coding agents from a single CLI command. 12 Codex Spark subagents. All running in parallel. At the same time. The project itself isn't complicated, but just imagine what this looks like when you scale the complexity. My setup: • Codex 5.3 High to plan • Codex 5.3 Medium to orchestrate • 12 Codex Spark agents (xhigh) to implement in one shot Spark isn't competing with the big models. It's a different tool entirely. You orchestrate it. You run it in swarms. And it moves at a speed that wasn't possible before, even for the top 1% of AI coders. More complex workflows coming soon. Comment below if you want me to tell you how I did it.

am.will

16,916 views • 7 months ago

GPT-Realtime 2 is the future of the operating system. I've been experimenting with it for a couple weeks now, and I gotta say, it's pretty gosh darn incredible. Opening apps, searching the web, even editing in Premiere. All with just my voice. And it only takes a few prompts to set up. In this video I'll show you exactly how. 0:00 Intro 1:46 What is GPT-Realtime 2? 4:46 Setting it up (no coding) 7:53 Fixing the always-on mic (push-to-talk) 9:41 Demo: searching the web 11:38 Demo: connecting apps via MCP (Obsidian) 14:13 Demo: controlling Premiere Pro (accessibility tree) 18:00 Honest caveats 19:00 Outro

GPT-Realtime 2 is the future of the operating system. I've been experimenting with it for a couple weeks now, and I gotta say, it's pretty gosh darn incredible. Opening apps, searching the web, even editing in Premiere. All with just my voice. And it only takes a few prompts to set up. In this video I'll show you exactly how. 0:00 Intro 1:46 What is GPT-Realtime 2? 4:46 Setting it up (no coding) 7:53 Fixing the always-on mic (push-to-talk) 9:41 Demo: searching the web 11:38 Demo: connecting apps via MCP (Obsidian) 14:13 Demo: controlling Premiere Pro (accessibility tree) 18:00 Honest caveats 19:00 Outro

Pat Simmons

607,765 views • 3 months ago

I joined OpenAI at the beginning of the year -- partly because I was excited about the possibility of better voice interaction with computers. So it was *especially* amazing to work with the team here on the gpt-4o model launch. It's hard to grok until you try it how big of a step forward this is for human/computer voice communication. It feels different -- it's obvious that the model understands you better than anything you've used before. This is my favorite demo of gpt-4o, it demonstrates not just good stt/tts, but actual conversational ability over full, realistic phone call. It's so obvious how useful this will be!

I joined OpenAI at the beginning of the year -- partly because I was excited about the possibility of better voice interaction with computers. So it was especially amazing to work with the team here on the gpt-4o model launch. It's hard to grok until you try it how big of a step forward this is for human/computer voice communication. It feels different -- it's obvious that the model understands you better than anything you've used before. This is my favorite demo of gpt-4o, it demonstrates not just good stt/tts, but actual conversational ability over full, realistic phone call. It's so obvious how useful this will be!

Peter Bakkum

144,441 views • 2 years ago

With all the buzz around the Codex App, OpenAI Developers quietly snuck out a new CLI update (0.94.0) as well. And boy is it an important update! Codex Plan mode is now officially released to the general audience! I am very excited about this one as it has a really strong prompt that is unlike any other plan mode I've personally used. Codex Plan mode doesn't necessarily just ask you 3 questions up front. It goes, collects context, asks questions, collects more context, asks more questions (sometimes), and then writes an incredibly high quality plan. It is my favorite implementation of plan mode thus far. It also comes with Codex's own version of "AskUserQuestion!" Although, it only works in Plan mode for now. They really need to allow people to use it in Code mode as well, but one win at a time. npm i -g @openai/codex Below is a demo of how it works. Let me know what you think!

With all the buzz around the Codex App, OpenAI Developers quietly snuck out a new CLI update (0.94.0) as well. And boy is it an important update! Codex Plan mode is now officially released to the general audience! I am very excited about this one as it has a really strong prompt that is unlike any other plan mode I've personally used. Codex Plan mode doesn't necessarily just ask you 3 questions up front. It goes, collects context, asks questions, collects more context, asks more questions (sometimes), and then writes an incredibly high quality plan. It is my favorite implementation of plan mode thus far. It also comes with Codex's own version of "AskUserQuestion!" Although, it only works in Plan mode for now. They really need to allow people to use it in Code mode as well, but one win at a time. npm i -g @openai/codex Below is a demo of how it works. Let me know what you think!

am.will

57,121 views • 7 months ago

GPT-6-Astra has solved Onshape by PTC for me 😎 it's been building a robot using the sensors i've chosen; i simply dictate to codex what changes i want, and then watch in real-time while it makes them happen in CAD this is ridiculous lol btw that's me controlling the mouse & model views while codex works from the backend using a connector that it built

GPT-6-Astra has solved Onshape by PTC for me 😎 it's been building a robot using the sensors i've chosen; i simply dictate to codex what changes i want, and then watch in real-time while it makes them happen in CAD this is ridiculous lol btw that's me controlling the mouse & model views while codex works from the backend using a connector that it built

alpha_rover

41,351 views • 12 days ago

It worked!! Kimi k3 Oauth is now wired directly into codex. The Kimi k3 model is insane but their app is not. So I figure a better way to use k3 by hooking my sub straight into codex instead. K3 now sits in the model picker next to Sol 5.6. Same subscription i was already paying for, no api key, just have to do OAuth in Kimi Code. Links down below 👇

It worked!! Kimi k3 Oauth is now wired directly into codex. The Kimi k3 model is insane but their app is not. So I figure a better way to use k3 by hooking my sub straight into codex instead. K3 now sits in the model picker next to Sol 5.6. Same subscription i was already paying for, no api key, just have to do OAuth in Kimi Code. Links down below 👇

Ziwen

276,639 views • 2 months ago

This is actually cool - I tried the same prompt for the new Interactive Playwright skill in Codex & GPT-5.4 xHigh - the one above is with the skill and the one below is without. What the skill does is uses the computer use capability of GPT-5.4 to look and navigate the UI. This never worked for me before, but with GPT-5.4 this is the first time I can actually see a massive difference. You can see how the first scene is much more coherent, higher fidelity and complete. The one below is missing a lot of elements and isn't as rich in detail. I'll keep using it for any UI work now.

This is actually cool - I tried the same prompt for the new Interactive Playwright skill in Codex & GPT-5.4 xHigh - the one above is with the skill and the one below is without. What the skill does is uses the computer use capability of GPT-5.4 to look and navigate the UI. This never worked for me before, but with GPT-5.4 this is the first time I can actually see a massive difference. You can see how the first scene is much more coherent, higher fidelity and complete. The one below is missing a lot of elements and isn't as rich in detail. I'll keep using it for any UI work now.

Peter Gostev (SF 24-28 August)

257,389 views • 6 months ago

"People assume prototyping requires writing code, but Image Gen is simply much faster." Rohan (Codex PM) showed me exactly how he does it: "I took a screenshot of the Codex composer where you can type in a new message. Then I asked Codex to use Image Gen to prototype 4-5 ideas to improve project selection. I can then ask it to take the first image and make a prototype of it on a Codex site. We usually think of Image Gen as useful for things like turning my hair blue. But it can create amazing digital mocks too." 📌 Full episode:

"People assume prototyping requires writing code, but Image Gen is simply much faster." Rohan (Codex PM) showed me exactly how he does it: "I took a screenshot of the Codex composer where you can type in a new message. Then I asked Codex to use Image Gen to prototype 4-5 ideas to improve project selection. I can then ask it to take the first image and make a prototype of it on a Codex site. We usually think of Image Gen as useful for things like turning my hair blue. But it can create amazing digital mocks too." 📌 Full episode:

Peter Yang

17,210 views • 2 months ago

haha the Runway real-time characters is great. i just spent 2 minutes pitching a ghibli version of myself a bladeless blender startup it's only a 2 min demo, but the questions and tone are pretty solid. one question even threw me off a bit (the cleaning). asking which round was unnecessary. it got a little weird at the end right before it cut off, which is due to a 2 min limit when testing on site(which seems unnecessary since they're charging me for the api - why not let me talk longer?) at current pricing, this would be roughly ~$6 for a 30 min call or $12 an hour. this doesn't include the cost of running an extraction prompt in the background to store this data, but llm tokens are so cheap. very expensive compared to mean vc (free, since it's in gpt marketplace) or even a self-hosted chat agent, but it's way cheaper than an associate. while it has downsides like not being a real person, there's also benefits like having consistent and unbiased information gathering, being available 24/7, being able to talk to founders in parallel, and consistent note taking. and it will likely get cheaper (sorry my audio volume is low, i'm testing while everyone else in the house is asleep)

haha the Runway real-time characters is great. i just spent 2 minutes pitching a ghibli version of myself a bladeless blender startup it's only a 2 min demo, but the questions and tone are pretty solid. one question even threw me off a bit (the cleaning). asking which round was unnecessary. it got a little weird at the end right before it cut off, which is due to a 2 min limit when testing on site(which seems unnecessary since they're charging me for the api - why not let me talk longer?) at current pricing, this would be roughly ~$6 for a 30 min call or $12 an hour. this doesn't include the cost of running an extraction prompt in the background to store this data, but llm tokens are so cheap. very expensive compared to mean vc (free, since it's in gpt marketplace) or even a self-hosted chat agent, but it's way cheaper than an associate. while it has downsides like not being a real person, there's also benefits like having consistent and unbiased information gathering, being available 24/7, being able to talk to founders in parallel, and consistent note taking. and it will likely get cheaper (sorry my audio volume is low, i'm testing while everyone else in the house is asleep)

Yohei

19,803 views • 6 months ago

I tried this so you don't have to. I know this is going to absolutely shock you but no this does not match the performance of Mythos. A few early thoughts: 1. The limits are pretty bad. I used 100% of my 5-hour usage in less than 1 prompt. 2. I specifically gave it a threejs task because it is an area that SOTA models have made big strides in, that other models just are not great at. I asked it to build a replica of Rocket League. I'll put the prompt in the comments. The game was pretty bad and notably worse than GPT 5.5. Even after multiple fixes, it took 7-8 back and forth with Codex just to get it an almost playable condition. Prior to these fixes, the game was not playable. Maybe it's really strong in other disciplines. I'd love to test that but I hit my limit in 1 prompt lol. GPT 5.5 by contrast did a pretty good job and required no follow ups. Fable would have absolutely nailed this as well. But yeah, early impressions...not great. But I hope I'm wrong. More testing tomorrow.

I tried this so you don't have to. I know this is going to absolutely shock you but no this does not match the performance of Mythos. A few early thoughts: 1. The limits are pretty bad. I used 100% of my 5-hour usage in less than 1 prompt. 2. I specifically gave it a threejs task because it is an area that SOTA models have made big strides in, that other models just are not great at. I asked it to build a replica of Rocket League. I'll put the prompt in the comments. The game was pretty bad and notably worse than GPT 5.5. Even after multiple fixes, it took 7-8 back and forth with Codex just to get it an almost playable condition. Prior to these fixes, the game was not playable. Maybe it's really strong in other disciplines. I'd love to test that but I hit my limit in 1 prompt lol. GPT 5.5 by contrast did a pretty good job and required no follow ups. Fable would have absolutely nailed this as well. But yeah, early impressions...not great. But I hope I'm wrong. More testing tomorrow.

am.will

594,106 views • 2 months ago