Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Side-by-side benchmarks beating OpenAI Codex computer-use using their own models! 👀 Round 1: Clip a youtube video from our channel and upload it to Tiktok ✅ + GPT 5.4 + our computer-use-kit: Successfully uploads a clip with subtitles and hook after 16 minutes (and works in iPhone/Android) ❌ Codex... + GPT 5.4: Gets the clip format wrong 3 times, asks for human intervention, and finally fails after 21 minutes. Codex actually does try iPhone mirroring and Capcut, which is very cool and kudos to the team, but it ultimately fails after burning credits. Sam Altman this is not easy to do, but happy to help you guys integrate our computer-use-kit. 😀 I co-founded the first startup approved by openAI to sell GPT3 for automation in August 2021(Cheatlayer) months before so I've been working on this for a long time. We automate Mac/Windows/Linux/Chrome/Android/Iphone + Daytona sandboxes and Browserbase cloud browsers out of the box. We also just shipped automated benchmarks, so we're building the most comprehensive computer-use benchmark for long-running tasks on the planet.show more

Rohan Arun

3,349 subscribers

120,579 Aufrufe • vor 6 Monaten •via X (Twitter)

Bildung Nachrichten & Politik Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

36 Kommentare

Profilbild von Bob Dylan

Bob Dylanvor 6 Monaten

@OpenAI Codex showed intent with CapCut + iPhone mirroring — but intent without recovery just burns credits. The retry layer on long-horizon tasks is the hardest part. Looks like you solved it.

Profilbild von Marc Almond

Marc Almondvor 6 Monaten

@OpenAI "Wrong format 3x, asks for help, fails" is most agents in prod — they just don't show you. Publishing the L side-by-side is a different move. Whoever owns the eval owns the narrative.

Profilbild von Retard Defi

Retard Defivor 6 Monaten

@OpenAI Real benchmark > demo reel. 16 min with subtitles, hook, and cross-platform support vs 21 min of retries and a failure. The gap speaks for itself. 👀

Profilbild von Web3_monarch

Web3_monarchvor 6 Monaten

@OpenAI GPT-3 automation in 2021, before anyone knew what to call it. The benchmark format is the right call — real tasks, real failure modes, no hiding. Curious if these go public.

Profilbild von Rohan Arun

Rohan Arunvor 6 Monaten

@GoogleDeepMind @AnthropicAI @deepseek_ai @alibaba_cloud @Meta happy to benchmark your best models too using our computer-use-kit!

Profilbild von xayintern

xayinternvor 6 Monaten

@OpenAI Codex: ❌ fails after 21 mins + human intervention Long-running agent workflows just got a new benchmark king. Rohan has been cooking since 2021. Respect.

Profilbild von Jongmin Park

Jongmin Parkvor 6 Monaten

@OpenAI lol Codex trying iPhone mirroring and Capcut on its own is honestly impressive even if it failed.

Profilbild von Oliver Dave

Oliver Davevor 6 Monaten

This is actually more important than most people realize Measuring output is one thing… but evaluating execution across long running agents introduces a completely different layer Things like state tracking, retries, UI changes, and multi step chaining all become real challenges Interesting to see this gap in execution play out here Curious to see how these benchmarks evolve

Profilbild von YabsssAI

YabsssAIvor 6 Monaten

@OpenAI what specific optimization did you make to get the clip uploaded with subtitles and a hook in just 16 seconds, was it mainly due to the computer-use-kit or some tweak in the GPT 5.4 integration

Profilbild von Rohan Arun

Rohan Arunvor 6 Monaten

@OpenAI Computer-use-kit since this can work with any model, including Nvidia Nemotron Super 3 and Nous Hermes 405B Instruct. We're publishing a white paper soon.

Profilbild von Ca-rdy🧩

Ca-rdy🧩vor 6 Monaten

@OpenAI Wish do my best as always

Profilbild von Kronic

Kronicvor 6 Monaten

@OpenAI It’s just impressive Too good

Profilbild von Web doctor

Web doctorvor 6 Monaten

@OpenAI The gap isn’t intelligence — it’s reliability. If it can’t complete the task without breaking or asking for help, it’s not ready yet. Multi-step workflows will expose this even more.

Profilbild von Duckworth

Duckworthvor 6 Monaten

@OpenAI A global run! Amazing

Profilbild von maryam

maryamvor 6 Monaten

@OpenAI Thats insane man!!! Lets connect

Profilbild von JĄVĄRI

JĄVĄRIvor 6 Monaten

@OpenAI Wow this is exceptional

Profilbild von sasuke.sol🥷🏾㊗️

sasuke.sol🥷🏾㊗️vor 6 Monaten

@OpenAI Exceptional ✨✨✨✨

Profilbild von No-B 👽🧩

No-B 👽🧩vor 6 Monaten

@OpenAI 🔥

Profilbild von King David🐺

King David🐺vor 6 Monaten

@OpenAI Love what you have been building Do hope we would work together someday.

Profilbild von Vito

Vitovor 6 Monaten

@OpenAI Amazing job Rohan 👏🏻

Profilbild von HolyGeeX

HolyGeeXvor 6 Monaten

@OpenAI This is so interesting

Profilbild von Blossom 🌸

Blossom 🌸vor 6 Monaten

@OpenAI That’s actually impressive

Profilbild von frostfox

frostfoxvor 6 Monaten

@OpenAI Amazing You worked really hard to achieve this

Profilbild von Rosinante

Rosinantevor 6 Monaten

@OpenAI Ne t’arrête pas!!🔥

Profilbild von ᴡᴇllɀ ×̷̷͜×̷

ᴡᴇllɀ ×̷̷͜×̷vor 6 Monaten

@OpenAI oh this is impressive

Profilbild von S. Bonkman-Fried

S. Bonkman-Friedvor 6 Monaten

@OpenAI you’re shipping nonstop 💪

Profilbild von Nik FX🧲

Nik FX🧲vor 6 Monaten

@OpenAI thats amazing

Profilbild von Alexa | Startup founder

Alexa | Startup foundervor 6 Monaten

@OpenAI Nuts how fast this is moving

Profilbild von Big Dave💊

Big Dave💊vor 6 Monaten

@OpenAI Quite impressive

Profilbild von Sadique — Ghost Startups

Sadique — Ghost Startupsvor 6 Monaten

@OpenAI this is impressive but curious are people actually using it in production or still early

Profilbild von Rohan Arun

Rohan Arunvor 6 Monaten

@OpenAI Yep we’re in the top 13 global Open Router personal agents in terms of token usage now

Profilbild von 𝐂𝐇𝐀𝐃 𝐃

𝐂𝐇𝐀𝐃 𝐃vor 6 Monaten

@OpenAI No explanation needed here, the future is here.

Profilbild von nelan

nelanvor 6 Monaten

@OpenAI Love this Rohan you cooked

Profilbild von Vivek Dhungav

Vivek Dhungavvor 6 Monaten

@OpenAI Execution is improving fast, but reliability is still the real test.

Profilbild von sasuke.sol🥷🏾㊗️

sasuke.sol🥷🏾㊗️vor 6 Monaten

@OpenAI Check here

Profilbild von sasuke.sol🥷🏾㊗️

sasuke.sol🥷🏾㊗️vor 6 Monaten

@OpenAI

Ähnliche Videos

WOW! I'm so excited about this. OpenAI Developers said Codex was good at Computer Use, but I wasn't prepared for this. For the last two weeks I've been working on a Computer Use skill to work with Linux. And while I had some success, it was a pretty frustrating experience. That is...until the breakthrough. Using accessibility tools, Codex can now control my entire computer, not just the browser. There are limits to this, of course, but what a time to be alive. This Computer Use skill will unlock and entirely new set of automations, all powered by Codex. Demonstration below 👇

WOW! I'm so excited about this. OpenAI Developers said Codex was good at Computer Use, but I wasn't prepared for this. For the last two weeks I've been working on a Computer Use skill to work with Linux. And while I had some success, it was a pretty frustrating experience. That is...until the breakthrough. Using accessibility tools, Codex can now control my entire computer, not just the browser. There are limits to this, of course, but what a time to be alive. This Computer Use skill will unlock and entirely new set of automations, all powered by Codex. Demonstration below 👇

am.will

79,698 Aufrufe • vor 7 Monaten

This is actually cool - I tried the same prompt for the new Interactive Playwright skill in Codex & GPT-5.4 xHigh - the one above is with the skill and the one below is without. What the skill does is uses the computer use capability of GPT-5.4 to look and navigate the UI. This never worked for me before, but with GPT-5.4 this is the first time I can actually see a massive difference. You can see how the first scene is much more coherent, higher fidelity and complete. The one below is missing a lot of elements and isn't as rich in detail. I'll keep using it for any UI work now.

This is actually cool - I tried the same prompt for the new Interactive Playwright skill in Codex & GPT-5.4 xHigh - the one above is with the skill and the one below is without. What the skill does is uses the computer use capability of GPT-5.4 to look and navigate the UI. This never worked for me before, but with GPT-5.4 this is the first time I can actually see a massive difference. You can see how the first scene is much more coherent, higher fidelity and complete. The one below is missing a lot of elements and isn't as rich in detail. I'll keep using it for any UI work now.

Peter Gostev

257,606 Aufrufe • vor 7 Monaten

Just found an open-source tool that lets you see how well an AI can actually use a computer. it’s called OSWorld 2.0. you give the AI a real task on a computer, then watch to see if it can figure everything out and finish it. it gives you: → real computer tasks, not simple questions → tasks that can take many steps to finish → the same computer setup for every AI → websites made for testing → all the files and tasks for free this is also the environment used to test computer-use systems from OpenAI and Anthropic. so if you’re building AI agents, researching computer use, or just want to see what these models can actually do, you can run the same tests yourself instead of relying only on benchmark scores. the labs publish the scores. this repo is where the actual tests come from. bookmark this.

Just found an open-source tool that lets you see how well an AI can actually use a computer. it’s called OSWorld 2.0. you give the AI a real task on a computer, then watch to see if it can figure everything out and finish it. it gives you: → real computer tasks, not simple questions → tasks that can take many steps to finish → the same computer setup for every AI → websites made for testing → all the files and tasks for free this is also the environment used to test computer-use systems from OpenAI and Anthropic. so if you’re building AI agents, researching computer use, or just want to see what these models can actually do, you can run the same tests yourself instead of relying only on benchmark scores. the labs publish the scores. this repo is where the actual tests come from. bookmark this.

MIKE

14,387 Aufrufe • vor 1 Monat

Databricks is excited to partner with OpenAI on GPT-5.5, their latest frontier model. GPT-5.5 will be available in Unity AI Gateway on launch. You can use it with coding tools such as Codex, or to power your enterprise agents. GPT-5.5 is state-of-the-art on many benchmarks including OfficeQA Pro, our benchmark for evaluating grounded reasoning on enterprise tasks. We are partnering with OpenAI to co-launch on Databricks. Hear more from our co-founder Patrick Wendell and OpenAI CRO Denise Holland Dresser on GPT-5.5 in Databricks.

Databricks is excited to partner with OpenAI on GPT-5.5, their latest frontier model. GPT-5.5 will be available in Unity AI Gateway on launch. You can use it with coding tools such as Codex, or to power your enterprise agents. GPT-5.5 is state-of-the-art on many benchmarks including OfficeQA Pro, our benchmark for evaluating grounded reasoning on enterprise tasks. We are partnering with OpenAI to co-launch on Databricks. Hear more from our co-founder Patrick Wendell and OpenAI CRO Denise Holland Dresser on GPT-5.5 in Databricks.

Databricks

12,917 Aufrufe • vor 6 Monaten

I asked jason (DevEx at OpenAI): “What’s the craziest thing Codex did for you?” Here’s his answer: “I was on a bike ride when a coworker asked me to fix a launch video. So I connected remotely from my phone and asked Codex to use computer use to edit the video, export it, and post it back to Slack. Twenty minutes later, there was a new video in the Slack thread. Then I told Codex: ‘Every 30 minutes, check the thread for feedback and export V2, V3, V4.’ By the time I got home, the launch video was greenlit.” Codex had handled the revisions by repeatedly checking Slack for feedback. 📌 Watch the full episode here:

I asked jason (DevEx at OpenAI): “What’s the craziest thing Codex did for you?” Here’s his answer: “I was on a bike ride when a coworker asked me to fix a launch video. So I connected remotely from my phone and asked Codex to use computer use to edit the video, export it, and post it back to Slack. Twenty minutes later, there was a new video in the Slack thread. Then I told Codex: ‘Every 30 minutes, check the thread for feedback and export V2, V3, V4.’ By the time I got home, the launch video was greenlit.” Codex had handled the revisions by repeatedly checking Slack for feedback. 📌 Watch the full episode here:

Peter Yang

79,504 Aufrufe • vor 2 Monaten

Nobody seems to know how insane GPT-5.4 is with computer use. I asked GPT-5.4 to draw the OpenAI logo in Microsoft Paint. No computer use API. Just a screenshot and basic tool calls (click, drag, press_key) all coordinate-based. The first drawing was awful. And GPT knew it. It looked at its own result and essentially went "yeah, no." What happened next is what broke my brain: It opened a browser. Went to Bing Images. Searched for the OpenAI logo. Found one. Then (and I cannot stress this enough) it used the Windows area screenshot shortcut (Win+Shift+S) to snip just the logo off the screen. Went back to Paint. Imported it. Centered it. All on its own. No instructions to do any of that. It just improvised a better strategy when the first one failed. My prompt was "Draw the OpenAI logo" with Paint already opened on the computer. Sure, it's "cheating." But honestly? That's exactly what I'd do too. And the fact that it came up with this plan from nothing but a screenshot and a coordinate system is wild.

Clad3815

1,070,585 Aufrufe • vor 7 Monaten

yesterday i showed claude code using codex's computer use, today i got some more fucking fire for you claude can also drive codex's official chrome extension, and it controls chromium browsers beautifully all in the background, without the "allow" popup chrome devtools mcp throws at you every time a new agent connects it even handles more than one browser: each install of the extension gets its own instance id, so claude knows which is which i have helium as my main and chrome for work, and claude code now uses both first, install the chrome plugin in the codex app and add openai's chatgpt extension to your chromium browser (or all of them, if you have a few) then give this to your claude: "Set up Codex's Chrome extension for you on my Mac. I've installed the Chrome plugin in the Codex app and the extension. Use Codex's computer use MCP server as user MCP "codex-cu": keep it if it's already registered, otherwise copy the "cua_repl" entry from ~/.codex/plugins/cache/openai-bundled/unified-computer-use/ /.mcp.json. Its browser side rejects calls without Codex turn metadata, so put a small stdio proxy in front that adds _meta["x-codex-turn-metadata"] (session_id and turn_id as a JSON string) to every tools/call. Add a Stop hook that tells this session's proxy to call the hidden turn_ended tool and start a new turn_id, so your tabs get released without touching other sessions. Test it by opening a tab straight at a URL in each browser with cua.createBrowserTab(browserId, url). Browsers can all report as "Chrome", so if I have several, ask me which test tab showed up in which browser." got more than one browser and ok with it touching your claude.md? give it this too: "Write each browser's name and metadata.extensionInstanceId into my global CLAUDE.md, with a note to pick browsers by that id and open tabs straight at a URL."

yesterday i showed claude code using codex's computer use, today i got some more fucking fire for you claude can also drive codex's official chrome extension, and it controls chromium browsers beautifully all in the background, without the "allow" popup chrome devtools mcp throws at you every time a new agent connects it even handles more than one browser: each install of the extension gets its own instance id, so claude knows which is which i have helium as my main and chrome for work, and claude code now uses both first, install the chrome plugin in the codex app and add openai's chatgpt extension to your chromium browser (or all of them, if you have a few) then give this to your claude: "Set up Codex's Chrome extension for you on my Mac. I've installed the Chrome plugin in the Codex app and the extension. Use Codex's computer use MCP server as user MCP "codex-cu": keep it if it's already registered, otherwise copy the "cua_repl" entry from ~/.codex/plugins/cache/openai-bundled/unified-computer-use/ /.mcp.json. Its browser side rejects calls without Codex turn metadata, so put a small stdio proxy in front that adds _meta["x-codex-turn-metadata"] (session_id and turn_id as a JSON string) to every tools/call. Add a Stop hook that tells this session's proxy to call the hidden turn_ended tool and start a new turn_id, so your tabs get released without touching other sessions. Test it by opening a tab straight at a URL in each browser with cua.createBrowserTab(browserId, url). Browsers can all report as "Chrome", so if I have several, ask me which test tab showed up in which browser." got more than one browser and ok with it touching your claude.md? give it this too: "Write each browser's name and metadata.extensionInstanceId into my global CLAUDE.md, with a note to pick browsers by that id and open tabs straight at a URL."

🥔🥔🥔

73,991 Aufrufe • vor 17 Tagen

building a $1M business for the second time. i'm launching a new product today! product number 5 since i started building again. to be honest, i'm a little nervous for this one because i put in a lot of work into it. it's a platform where you can create AI employees and treat them like actual human remote employees. > you hire/create them > you give them their own computer > you help them set up their computer by installing apps and logging into services > you assign tasks and TODOs for them > and then let them perform the tasks under the hood they use Codex and Claude Code, paired with computer use and tool use. so they can connect to any tool and perform literally any task, as long as they have instructions. > coding > marketing > customer support > client communication > general assistance work > and anything else you want hope you like it! btw, this is also the best domain i own ❤️

building a $1M business for the second time. i'm launching a new product today! product number 5 since i started building again. to be honest, i'm a little nervous for this one because i put in a lot of work into it. it's a platform where you can create AI employees and treat them like actual human remote employees. > you hire/create them > you give them their own computer > you help them set up their computer by installing apps and logging into services > you assign tasks and TODOs for them > and then let them perform the tasks under the hood they use Codex and Claude Code, paired with computer use and tool use. so they can connect to any tool and perform literally any task, as long as they have instructions. > coding > marketing > customer support > client communication > general assistance work > and anything else you want hope you like it! btw, this is also the best domain i own ❤️

Alex West 🚀

20,575 Aufrufe • vor 1 Monat

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Andrew Ng

170,647 Aufrufe • vor 1 Jahr

🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: Leaderboard (tentative): Blog: Data & Code (coming soon): ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]

🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: Leaderboard (tentative): Blog: Data & Code (coming soon): ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]

Bowen Wang

93,330 Aufrufe • vor 1 Jahr

Introducing Tapkit Tapkit is a Mac app that lets your agents use your iPhone. Tell it to do anything and it can use your phone to do it. We've been iterating on Tapkit over the last few months with our earliest users and finally feel like it's ready for general use. Today it is easy to use, supports any agent, and can even use multiple iPhones at the same time. There are so many great things that we can do with this new primitive, but I didn't want to wait any longer to share it with you. Try it now:

Introducing Tapkit Tapkit is a Mac app that lets your agents use your iPhone. Tell it to do anything and it can use your phone to do it. We've been iterating on Tapkit over the last few months with our earliest users and finally feel like it's ready for general use. Today it is easy to use, supports any agent, and can even use multiple iPhones at the same time. There are so many great things that we can do with this new primitive, but I didn't want to wait any longer to share it with you. Try it now:

teej

576,141 Aufrufe • vor 23 Tagen

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

Matthew Lam

17,674 Aufrufe • vor 5 Monaten

Sam Altman says OpenAI killed two shipped products, Sora and the Atlas browser, and names what each one cost the company: "But last year, for example, we killed Sora, which was, you know, good product and fun and cool" "but used a lot of compute and not as important as Codex, where we put the compute." "We killed our web browser called Atlas, which, again, I think it was a great product." "I think it was the best web browser, but not as important for us to focus on..." Sora's app and web experiences were discontinued in April, and Atlas stopped working on August 9 with its browser agent work moved into ChatGPT and Codex. He names a different constraint behind each decision, compute in the first case and where the team spends its attention in the second. - Sam Altman (Sam Altman), CEO of OpenAI, with David Senra (David Senra).

Sam Altman says OpenAI killed two shipped products, Sora and the Atlas browser, and names what each one cost the company: "But last year, for example, we killed Sora, which was, you know, good product and fun and cool" "but used a lot of compute and not as important as Codex, where we put the compute." "We killed our web browser called Atlas, which, again, I think it was a great product." "I think it was the best web browser, but not as important for us to focus on..." Sora's app and web experiences were discontinued in April, and Atlas stopped working on August 9 with its browser agent work moved into ChatGPT and Codex. He names a different constraint behind each decision, compute in the first case and where the team spends its attention in the second. - Sam Altman (Sam Altman), CEO of OpenAI, with David Senra (David Senra).

Karl Mehta

25,306 Aufrufe • vor 1 Monat

Today we're excited to introduce Vy, our AI that sees and acts on your computer. At Vercept, our mission is to reinvent how humans use computers–enabling you to accomplish orders of magnitude more than what you can do today. Vy is a first glimpse at AI that sees and uses your computer just like you do. It runs natively on your computer, with access to all your applications and accounts that you’re already signed into. It is available for download starting today! As part of the release, we’ve built VyUI, an AI model bridging the gap between language and your screen. It sets a new state-of-the-art on UI grounding benchmarks like ScreenSpot v1, ScreenSpot v2, and GroundUI Web–beating leading models from OpenAI, Google, and Anthropic. Check out what Vy can do in the thread below: 🧵👇

Today we're excited to introduce Vy, our AI that sees and acts on your computer. At Vercept, our mission is to reinvent how humans use computers–enabling you to accomplish orders of magnitude more than what you can do today. Vy is a first glimpse at AI that sees and uses your computer just like you do. It runs natively on your computer, with access to all your applications and accounts that you’re already signed into. It is available for download starting today! As part of the release, we’ve built VyUI, an AI model bridging the gap between language and your screen. It sets a new state-of-the-art on UI grounding benchmarks like ScreenSpot v1, ScreenSpot v2, and GroundUI Web–beating leading models from OpenAI, Google, and Anthropic. Check out what Vy can do in the thread below: 🧵👇

Vercept

123,859 Aufrufe • vor 1 Jahr

GPT-6 Astra “uses Unity to assemble a city scene from existing assets, allowing users to easily create an immersive 3D environment to explore that matches their vision.” “Alongside Astra, we are also updating the Codex harness to significantly improve the speed of computer use. Combined with Astra's efficiency, this translates to a 1.9x faster task completion compared to the current GPT-5.6 Sol experience, on the Mind2Web benchmark. “ “The model's improvements on speed mean it can take on many time consuming life tasks for you, faster than you can.*” I’ll admit, others and I have created far more impressive demos, but keep in mind this was done in Unity, which is a much more difficult environment for models to work with. If they weren’t using computer use which would’ve been extremely slow they were likely manipulating the project programmatically through C# and/or Unity’s YAML-serialized scene files which again models are pretty bad at. So this is a solid upgrade from 5.6 - from what I’ve seen unity!

GPT-6 Astra “uses Unity to assemble a city scene from existing assets, allowing users to easily create an immersive 3D environment to explore that matches their vision.” “Alongside Astra, we are also updating the Codex harness to significantly improve the speed of computer use. Combined with Astra's efficiency, this translates to a 1.9x faster task completion compared to the current GPT-5.6 Sol experience, on the Mind2Web benchmark. “ “The model's improvements on speed mean it can take on many time consuming life tasks for you, faster than you can.*” I’ll admit, others and I have created far more impressive demos, but keep in mind this was done in Unity, which is a much more difficult environment for models to work with. If they weren’t using computer use which would’ve been extremely slow they were likely manipulating the project programmatically through C# and/or Unity’s YAML-serialized scene files which again models are pretty bad at. So this is a solid upgrade from 5.6 - from what I’ve seen unity!

Chris

22,655 Aufrufe • vor 1 Monat

Sam Altman reveals building Codex was a "crazy kamikaze mission" to beat Claude Code and that most of the best coders he knows now use Codex. "We were way behind Claude Code, and it seemed like a kind of crazy kamikaze mission to try to beat them with a coding app." "The consensus is that this kind of thing never works, and you just move on to the next one. It's a fool's errand to try to win when someone else already has momentum in a particular product category." "But we decided we thought it was really important. We asked a team to do it, and they performed a legitimate, unbelievable, very rare in the history of business thing." "Now it is the product that most of the best coders I know use." "That felt like an impossible thing, but if we hadn't asked the team, hey, we have a really important but extremely hard mission for you, it just wouldn't have happened."

Sam Altman reveals building Codex was a "crazy kamikaze mission" to beat Claude Code and that most of the best coders he knows now use Codex. "We were way behind Claude Code, and it seemed like a kind of crazy kamikaze mission to try to beat them with a coding app." "The consensus is that this kind of thing never works, and you just move on to the next one. It's a fool's errand to try to win when someone else already has momentum in a particular product category." "But we decided we thought it was really important. We asked a team to do it, and they performed a legitimate, unbelievable, very rare in the history of business thing." "Now it is the product that most of the best coders I know use." "That felt like an impossible thing, but if we hadn't asked the team, hey, we have a really important but extremely hard mission for you, it just wouldn't have happened."

Fireside Alpha

2,154,837 Aufrufe • vor 2 Monaten

I’m reverse-engineering the Google AI mouse pointer and making it Open Source! It sees your screen. You can freeform paint anything and ask AI to modify it. It understands the app or window you mean. It can click, type, edit, navigate, and even write + execute code. A tiny cursor becoming a real computer-use agent. Made with the help of Cua which is an AMAZING reference for computer use Sorry Google DeepMind but I had to do this heyclicky Started a revolution and we will be the ones to win Also credits to Omkar Satpute for the help Feel free to modify it , update it and make it your own, we gotta beat them at this. Link to the repo in the comments:

I’m reverse-engineering the Google AI mouse pointer and making it Open Source! It sees your screen. You can freeform paint anything and ask AI to modify it. It understands the app or window you mean. It can click, type, edit, navigate, and even write + execute code. A tiny cursor becoming a real computer-use agent. Made with the help of Cua which is an AMAZING reference for computer use Sorry Google DeepMind but I had to do this heyclicky Started a revolution and we will be the ones to win Also credits to Omkar Satpute for the help Feel free to modify it , update it and make it your own, we gotta beat them at this. Link to the repo in the comments:

Milind S

215,990 Aufrufe • vor 5 Monaten

When I talk to serious software engineers, they tell me everyone is sleeping on Codex + GPT 5.2 from OpenAI Which is why I was hyped to get an insiders "how to" from Alexander Embiricos product lead for Codex, showing us how he and the team use Codex to ship almost everything at oai. He'll show us how to: - jump into a new repo and start contributing, even as a non-technical PM - run git worktrees to parallelize agent tasks across codex instances - use best practices and to spec out your work and let codex rip We also talk model vs. harness and how to spy on codex development (which is all OSS!) to understand how to optimize your own implementations. This episode is brought to you by our amazing sponsors 🧠🤑 Brex - The intelligent finance platform built for founders 🦎🐛 Greptile - Your AI code review platform Watch it here:

When I talk to serious software engineers, they tell me everyone is sleeping on Codex + GPT 5.2 from OpenAI Which is why I was hyped to get an insiders "how to" from Alexander Embiricos product lead for Codex, showing us how he and the team use Codex to ship almost everything at oai. He'll show us how to: - jump into a new repo and start contributing, even as a non-technical PM - run git worktrees to parallelize agent tasks across codex instances - use best practices and to spec out your work and let codex rip We also talk model vs. harness and how to spy on codex development (which is all OSS!) to understand how to optimize your own implementations. This episode is brought to you by our amazing sponsors 🧠🤑 Brex - The intelligent finance platform built for founders 🦎🐛 Greptile - Your AI code review platform Watch it here:

claire vo 🖤

48,134 Aufrufe • vor 9 Monaten

I think a lot of people are missing what GPT-6 can really do because they're using it out of the box. Give it the right free MCPs and skills, and it starts to feel like magic. Start with Blender MCP. In my tests, it works faster and uses fewer tokens than computer use. If you want to have fun with procedural modeling, go for it. For more detailed models, I use 3D generators and let GPT handle the processing, rigging and animation. You don't need to burn tokens making it build every detail from scratch. I've spent the last six months putting this setup together. These are the tools I actually use for Blender, Unity, Godot and Unreal, including my 3D generation tools. Full breakdown:

I think a lot of people are missing what GPT-6 can really do because they're using it out of the box. Give it the right free MCPs and skills, and it starts to feel like magic. Start with Blender MCP. In my tests, it works faster and uses fewer tokens than computer use. If you want to have fun with procedural modeling, go for it. For more detailed models, I use 3D generators and let GPT handle the processing, rigging and animation. You don't need to burn tokens making it build every detail from scratch. I've spent the last six months putting this setup together. These are the tools I actually use for Blender, Unity, Godot and Unreal, including my 3D generation tools. Full breakdown:

Stefan 3D AI

26,513 Aufrufe • vor 1 Monat