🚀 Introducing 𝐀𝐠𝐞𝐧𝐭 𝐒3, the most advanced computer-use agent, now 𝐚𝐩𝐩𝐫𝐨𝐚𝐜𝐡𝐢𝐧𝐠 𝐡𝐮𝐦𝐚𝐧-𝐥𝐞𝐯𝐞𝐥 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞🧠💻 Just one year ago, Agent S scored ~20% on OSWorld: SOTA then, but far from human 72%. Today, Agent S3 reaches 6̳9̳.̳9̳%̳ (⬆10% over prior SOTA), nearly matching humans on computer use. Few imagined this gap... could close so fast, but with steady progress and 𝐟𝐢𝐫𝐬𝐭-𝐩𝐫𝐢𝐧𝐜𝐢𝐩𝐥𝐞 simplicity, Simular did it in just one year. 🧪 The s̵e̵c̵r̵e̵t̵ 𝐨𝐩𝐞𝐧 𝐫𝐞𝐜𝐢𝐩𝐞: → 𝐁𝐞𝐡𝐚𝐯𝐢𝐨𝐫 𝐁𝐞𝐬𝐭-𝐨𝐟-𝐍 — scaling agents 𝘵𝘩𝘦 𝘳𝘪𝘨𝘩𝘵 𝘸𝘢𝘺 → Simpler design, stronger baseline → 100% 𝐨𝐩𝐞𝐧 𝐬𝐨𝐮𝐫𝐜𝐞 (as always) Learn more at & 👇show more

🧵1/N: New SOTA, Near Human-level Performance On OSWorld Verified, Agent S3 alone (no scaling) reaches 62.6% in the 100-step setting, already exceeding the previous SOTA of 59.9%. With the addition of Behavior Best-of-N, performance climbs even higher to 69.9%, bringing computer-use agents to within just 2% of human-level accuracy (72%).

🧵2/N: CUA Bottleneck: High Variance and Inconsistency Computer-use agents (CUAs) promise a future where software runs itself, booking tickets, filling forms, and navigating apps so you don’t have to. But right now, even the best CUAs stumble when tasks get long and messy. A stray click, a late response, or an unexpected pop-up can send the whole run off course. Small mistakes compound, and what should have been smooth automation turns into frustration. That’s the core bottleneck: high variance. The same agent might nail a task once and then completely blow it the next time. This inconsistency makes CUAs unpredictable and shows why reliability on complex, everyday workflows remains such a challenge.

🧵3/N: Wide Scaling of CUA A natural way to mitigate this fragility is wide scaling: instead of simply accepting a single rollout from one agent, we can scale the number of agents to generate multiple rollouts in parallel and select the best. This wide scaling perspective leverages the fact that agents, while suboptimal individually, often succeed on complementary subsets of tasks However, scaling CUAs introduces unique challenges. 1⃣long-horizon trajectories are information-dense with multimodal details, most of which are irrelevant to task success, making them difficult to represent, interpret, and compare. 2⃣evaluation itself is non-trivial: many computer-use tasks admit multiple valid solutions, and automatic evaluation struggles to decide whether a trajectory is correct. Therefore, effective wide scaling of CUAs requires new methods that - (1) compactly encode long trajectories while preserving task-relevant semantics, - and (2) reliably evaluate diverse solution paths to identify successful rollouts.

🧵4/N: Behavior Best-of-N Enables Wide Scaling of CUAs Agent S3 introduces Behavior Best-of-N (bBoN) to address those challenges. (1) Behavior Narrative Generation Our approach starts by generating facts. Raw agent runs contain a large amount of step-by-step detail, much of which is irrelevant or redundant. By generating facts, we convert these noisy runs into concise statements about what happened at each step, focusing only on the information that directly matters for task success. Concatenating these facts produces a behavior narrative, which is a clear summary of what an agent did at each step, making agent runs more interpretable and easier to compare. (2) Behavior Best-of-N Judge With behavior narratives in place, we apply judge selection to determine which rollout best completes the task. Instead of comparing raw outputs, the judge grounds its decision in the facts within each behavior narrative. By citing these facts across rollouts, the judge can reason comparatively about which attempt is most effective and ultimately selects the best run.

🧵5/N: Improved Framework Baseline: Simpler Design, Greater Flexibility Agent S2 used a hierarchical manager–worker setup, but this has become unnecessary overhead as the foundation models get stronger. Agent S3 streamlines the framework by removing that hierarchy and introducing a native coding agent that can generate and execute code. This makes solutions more diverse, spanning both code and GUI tasks, and also more reliable. Together, these refinements boosted performance by about 13%, bringing Agent S3 to 62.6% for single-run performance (already SOTA).

🧵6/N: How does our method perform? Building on top of the improved baseline, our core contribution, Behavior Best-of-N (bBoN), achieves 69.9% SR with GPT-5 (a 7.3% absolute improvement over Agent S3 single run) and 60.2% SR with GPT-5 Mini (a 10.4% absolute improvement). Given that human performance is approximately 72%, these results highlight that bBoN not only surpasses existing methods by a large margin but also approaches near human-level capability.

🧵7/N: How does our method perform? (cont.) In addition, we also compare Agent S3 with Agent S2, on the single run setting (without scaling). Behavior Best-of-N yields a 13.8% improvement in success rate, a 52.3% reduction in LLM calls per task, and a 62.4% reduction in average task completion time. Agent S3 is both more capable and more efficient.

🧵8/N: Scaling with Agent Runs As we increase the number of agent runs on OSWorld, we find that performance gradually improves. With 10 parallel agent runs, it achieves the highest performance with GPT-5 at 69.9% and with GPT-5 Mini at 60.2%.

🧵9/N: How should we select a Mixture-of-Models Ensemble? bBoN can scale across different models. Table below shows the success rate and task coverage of bBoN using various mixture-of-model combinations. Task coverage is calculated by setting a task successful if at least one trajectory is correct, or Pass@N. We observe that: - The GPT-5 + Gemini 2.5 Pro mixture achieves the highest success rate of 66.7% and task coverage of 78.0%, suggesting that selecting a mixture-of-models ensemble with highly diverse capable models achieves the best performance with the highest upper bound. - the most diverse mixture (All) achieves higher task coverage than single-model mixtures at 75.4%, demonstrating that diversity is key to increasing the upper bound on success. - the GPT-5 + Gemini 2.5 Pro mixture achieves the highest success rate of 66.7% and task coverage of 78.0%, suggesting that selecting a mixture-of-models ensemble with highly diverse capable models achieves the best performance with the highest upper bound.

🧵10/N: How Behavior Narratives Compare to Other Trajectory Representations? We show an ablation on our behavior narrative representation. We compare against a screenshot-only baseline and a naive captioning baseline that captions each screenshot individually. We find that behavior narratives are an effective representation for bBoN, providing a 3.4% improvement over the best baseline. This suggests that it is difficult to understand screenshots alone and that it is necessary to generate facts over transitions rather than individual states.

🧵11/N: Judge Comparison We show a comparison between bBoN Judge and WebJudge. We modify WebJudge to choose over many trajectories by independently ranking trajectories and selecting the highest rank. Overall, bBoN Judge achieves better performance than WebJudge, with WebJudge providing limited benefit over the average performance of rollouts. We also find that our judge shows better scaling as we increase the number of rollouts.

🧵12/N: Judge Alignment with Human We took a look at tasks where our judge could improve performance (44% of OSWorld) and found that the judge chose correctly for 78.4% of tasks. When we double-checked, we found that the judge was actually correct for 92.8% of tasks, making the true performance on OSWorld closer to 76.3%. This suggests our judge aligns well with human preferences, making it a promising tool for evaluating CUA tasks.

🧵13/N: Generalization to Other Benchmarks / Systems Our method demonstrates strong generalizability to different operating systems. For AndroidWorld, we compare with the top 3 performing open-source, screenshot-only methods, including AgentS2, MobileUse, and UI-Venus. For WindowsAgentArena, we compare with Agent S2 and UI-TARS-1.5. Our bBoN method with N = 3 achieves a performance boost of 3.5% and 6.4% respectively, demonstrating that our method can generalize well to other domains.

🧵 14/14: Everything is Open Source! 📄 Paper: 💻 Code: 📝 Blog: Next up: super-human computer use! 🚀💻✨

hi there is CTA button that does nothing

Fixed. Thanks!

@Scobleizer How to try? Any demo available?

@Scobleizer It is open sourced:

The jump from 20% → 70% in a year is crazy. Behavior Best-of-N really makes these agents way more consistent.

@angli_ai insane excited to try it, congrats on the release!

That’s insane progress in just a year, closing the gap with humans way faster than anyone expected.

Impressive

Incredible progress! Exciting times ahead for AI.

Impressive gains from Agent S to S3 in just one year.

Have you tested with Claude 4.5 Sonnet? I hear it's a great CUA.

~61%

Incredible progress! The future of AI looks bright.

looking forward to trying, thanks

Impressive progress! Agent S3 is closing the gap fast. Exciting times ahead!

Amazing inventions

Impressive progress! Agent S3 is truly remarkable. Keep up the great work!

Closing the gap from 20% to nearly human-level in a year is massive, computer-use agents are moving faster than most expected. The open-source angle could accelerate this even more as the community iterates. I share more thoughts on breakthroughs like this in my profile.

Wow. 69.9% is very impressive.

This is Amazing!

Impressive progress on Agent S3! Almost at human level now. Exciting advancements.

Loved it.

Huge leap in just a year—impressive progress!

Just launched and already near human-level, that's impressive!

Agent S3 hits 69.9% almost human at computer use.

Big congrats to the team on Agent S3

Amazing tool for sure

The new S3 version looks powerful

Nice

Open-source makes this even more exciting

Nice !

Amazing

As agents near human-level performance, the next big challenge is scaling them reliably in the real world. That’s where orchestration layers like Coral become essential.

Need to check this out.

Huge leap in just a year—impressive progress!

The simplicity-first approach really paid off here.