Meet Agent Bake-Off: blind side by side tests for coding agents like Claude Code, Cursor , Amp , Factory , Codex, & more. We've been building the largest real world benchmarks for coding agents, and we're excited share an early preview of our open community tooling.

Real-world use data on how agents are used & procured will become more important as agents go mainstream. We want anyone to be able to run fair comparisons, contribute to public leaderboards, & ground the ecosystem in real preferences. Try it here:

It’s super simple right now: - pick a pre-set public repo - enter a task - we run it on two anonymous agents with the same context Some tasks can take some time so you can optionally submit your email to get a ping when results are ready.

When it finishes you get: - side by side outputs and diffs - metadata like reasoning notes, tools used, and time spent - a blind vote to pick the better result Your votes and runs feed the public community leaderboards.

This is still a very early version, and we're excited to see what is useful to people. Some areas we're exploring are: what better UI/UX for reviewing parallelized agentic work looks like, what metadata and tasks is valuable, etc. *Especially* for real-world use

We're also going to be constantly updating Agent Bake-Off to add more frontier agents, other subsets of agents (PR review agents, security/guardrail agents, etc.), seeing if people want the ability to bring their own public/private repos, etc.

We are working hard to build the largest real world benchmarks and leaderboards for agents. Agent Bake-Off is a small first step. Please try it, vote, and give us feedback on what you would find useful!!!

@cursor_ai @AmpCode @FactoryAI omg you bake??

@cursor_ai @AmpCode @FactoryAI don't tell anyone

@sonofalli @cursor_ai @AmpCode @FactoryAI Let me know if you’re going to do a live bake-off agent coding event in SF. Kids and I will actually bake and show up to watch.

@sonofalli @cursor_ai @AmpCode @FactoryAI Will likely need to have real baked goods to get anyone to enjoy it!

@cursor_ai @AmpCode @FactoryAI 🔥

@0xnirmal @cursor_ai @AmpCode @FactoryAI Not as impressive as hardcore RL environments :)

@cursor_ai @AmpCode @FactoryAI 🔥🔥

@cursor_ai @AmpCode @FactoryAI we love powerful ephemeral sandboxes

@cursor_ai @AmpCode @FactoryAI Game on!

@cursor_ai @AmpCode @FactoryAI Congrats @brexton!

@cursor_ai @AmpCode @FactoryAI Powered by @e2b :)

@cursor_ai @AmpCode @FactoryAI Go go go!!!

@cursor_ai @AmpCode @FactoryAI thank you 😭

@cursor_ai @AmpCode @FactoryAI interesting! Factory seems like a winner on quality code and cost efficiency

@cursor_ai @AmpCode @FactoryAI Factory has been really good recently and they’re super generous with credits If you’re looking for super cost effective I would try Gemini CLI too!!

@cursor_ai @AmpCode @FactoryAI no for gemini too dumb. Cheap is cheap with pro 2.5. Waiting for Sir Demis to give us 3.0 ultra

@cursor_ai @AmpCode @FactoryAI Gonna be a power user of this 👀

@cursor_ai @AmpCode @FactoryAI :) let me know what other agents or other categories of agents (PR review, security, text-to-app, etc.) that you'd like to see!

@cursor_ai @AmpCode @FactoryAI Great work guys! Really happy to see this as a developer!

@cursor_ai @AmpCode @FactoryAI Please try it out and let us know what you think!

@cursor_ai @AmpCode @FactoryAI Super cool 😎

@cursor_ai @AmpCode @FactoryAI Would love your feedback on some other stuff we have cooking @morganlinton!

@cursor_ai @AmpCode @FactoryAI This is awesome, blind side-by-side tests and open benchmarks will shake things up can't wait to try the tools

@cursor_ai @AmpCode @FactoryAI try it and give us your feedback!

@cursor_ai @AmpCode @FactoryAI very cool!

@cursor_ai @AmpCode @FactoryAI 🍳

@cursor_ai @AmpCode @FactoryAI We are prepping a special credits system for @askModuAI for everyone building on @base

@cursor_ai @AmpCode @FactoryAI ‼️

@cursor_ai @AmpCode @FactoryAI i think we should host some holiday events together @MaxBrodeurUrbas

@cursor_ai @AmpCode @FactoryAI This is dope

@cursor_ai @AmpCode @FactoryAI Let's goo!

@cursor_ai @AmpCode @FactoryAI Thank you 😭

@cursor_ai @AmpCode @FactoryAI we beat them all...

@cursor_ai @AmpCode @FactoryAI Love it!

@cursor_ai @AmpCode @FactoryAI Amazing!

@cursor_ai @AmpCode @FactoryAI Brexton, that Agent Bake-Off sounds quite exciting, and these real-world benchmarks are a much-needed thing, no?

@cursor_ai @AmpCode @FactoryAI Very nice article! One company is the U.S. thats pioneering new additives for the chemical manufacturing companies is They’ve already created a new way of treat PFAS.