Загрузка видео...
Не удалось загрузить видео
Side-by-side benchmarks beating OpenAI Codex computer-use using their own models! 👀 Round 1: Clip a youtube video from our channel and upload it to Tiktok ✅ + GPT 5.4 + our computer-use-kit: Successfully uploads a clip with subtitles and hook after 16 minutes (and works in iPhone/Android) ❌ Codex... show more
120,579 просмотров • 6 месяцев назад •via X (Twitter)
Комментарии: 36
@OpenAI Codex showed intent with CapCut + iPhone mirroring — but intent without recovery just burns credits. The retry layer on long-horizon tasks is the hardest part. Looks like you solved it.
@OpenAI "Wrong format 3x, asks for help, fails" is most agents in prod — they just don't show you. Publishing the L side-by-side is a different move. Whoever owns the eval owns the narrative.
@OpenAI Real benchmark > demo reel. 16 min with subtitles, hook, and cross-platform support vs 21 min of retries and a failure. The gap speaks for itself. 👀
@OpenAI GPT-3 automation in 2021, before anyone knew what to call it. The benchmark format is the right call — real tasks, real failure modes, no hiding. Curious if these go public.
@GoogleDeepMind @AnthropicAI @deepseek_ai @alibaba_cloud @Meta happy to benchmark your best models too using our computer-use-kit!
@OpenAI Codex: ❌ fails after 21 mins + human intervention Long-running agent workflows just got a new benchmark king. Rohan has been cooking since 2021. Respect.
@OpenAI lol Codex trying iPhone mirroring and Capcut on its own is honestly impressive even if it failed.
This is actually more important than most people realize Measuring output is one thing… but evaluating execution across long running agents introduces a completely different layer Things like state tracking, retries, UI changes, and multi step chaining all become real challenges Interesting to see this gap in execution play out here Curious to see how these benchmarks evolve
@OpenAI what specific optimization did you make to get the clip uploaded with subtitles and a hook in just 16 seconds, was it mainly due to the computer-use-kit or some tweak in the GPT 5.4 integration
@OpenAI Computer-use-kit since this can work with any model, including Nvidia Nemotron Super 3 and Nous Hermes 405B Instruct. We're publishing a white paper soon.
@OpenAI Wish do my best as always
@OpenAI It’s just impressive Too good
@OpenAI The gap isn’t intelligence — it’s reliability. If it can’t complete the task without breaking or asking for help, it’s not ready yet. Multi-step workflows will expose this even more.
@OpenAI A global run! Amazing
@OpenAI Thats insane man!!! Lets connect
@OpenAI Wow this is exceptional
@OpenAI Exceptional ✨✨✨✨
@OpenAI 🔥
@OpenAI Love what you have been building Do hope we would work together someday.
@OpenAI Amazing job Rohan 👏🏻
@OpenAI This is so interesting
@OpenAI That’s actually impressive
@OpenAI Amazing You worked really hard to achieve this
@OpenAI Ne t’arrête pas!!🔥
@OpenAI oh this is impressive
@OpenAI you’re shipping nonstop 💪
@OpenAI thats amazing
@OpenAI Nuts how fast this is moving
@OpenAI Quite impressive
@OpenAI this is impressive but curious are people actually using it in production or still early
@OpenAI Yep we’re in the top 13 global Open Router personal agents in terms of token usage now
@OpenAI No explanation needed here, the future is here.
@OpenAI Love this Rohan you cooked
@OpenAI Execution is improving fast, but reliability is still the real test.
@OpenAI Check here
@OpenAI
![yesterday i showed claude code using codex's computer use, today i got some more fucking fire for you claude can also drive codex's official chrome extension, and it controls chromium browsers beautifully all in the background, without the "allow" popup chrome devtools mcp throws at you every time a new agent connects it even handles more than one browser: each install of the extension gets its own instance id, so claude knows which is which i have helium as my main and chrome for work, and claude code now uses both first, install the chrome plugin in the codex app and add openai's chatgpt extension to your chromium browser (or all of them, if you have a few) then give this to your claude: "Set up Codex's Chrome extension for you on my Mac. I've installed the Chrome plugin in the Codex app and the extension. Use Codex's computer use MCP server as user MCP "codex-cu": keep it if it's already registered, otherwise copy the "cua_repl" entry from ~/.codex/plugins/cache/openai-bundled/unified-computer-use/ /.mcp.json. Its browser side rejects calls without Codex turn metadata, so put a small stdio proxy in front that adds _meta["x-codex-turn-metadata"] (session_id and turn_id as a JSON string) to every tools/call. Add a Stop hook that tells this session's proxy to call the hidden turn_ended tool and start a new turn_id, so your tabs get released without touching other sessions. Test it by opening a tab straight at a URL in each browser with cua.createBrowserTab(browserId, url). Browsers can all report as "Chrome", so if I have several, ask me which test tab showed up in which browser." got more than one browser and ok with it touching your claude.md? give it this too: "Write each browser's name and metadata.extensionInstanceId into my global CLAUDE.md, with a note to pick browsers by that id and open tabs straight at a URL."](https://image.24vids.com/tw-2106789987907940538/amplify_video_thumb/2106784556720529408/img/ryyKFFArtD7vAWHT.jpg)
![🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: Leaderboard (tentative): Blog: Data & Code (coming soon): ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]](https://image.24vids.com/tw-1909618451259572328/ext_tw_video_thumb/1909618081078677504/pu/img/C6ALLNvR2qWTw6sI.jpg)
