Stop vibe checking your vibe code! We just released Vibe Code Bench: the first benchmark that tests whether AI models can actually build complete web applications from scratch. Featured today in Inc. (1/6)

Despite billions in investment, only 2 models show real capability: GPT 5.1 and Claude Sonnet 4.5 Thinking. Every other model falls below 15%. Most can't build working apps at all. (2/6)

We gave models 5 hours and a full dev environment to build full web applications. They had terminal access, a database, browser access, and could use any libraries of their choice. (3/6)

We noticed some common failure modes: - Installing dependencies wrong repeatedly -Forgetting key requirements mid-task -Spending dozens of turns stuck on Docker networking -Giving up early despite having time left -Only top models debugged efficiently. (4/6)

Existing benchmarks testing how AI assists developers, not whether models can deliver working software end-to-end. If AI can go from natural language to working software, it unlocks the power of software for everyone—not just engineers. (5/6)

Vibe Code Bench is live, check it out here: (6/6)

@Inc This is sick, are you planning to add agents (Devin, Cursor, etc) to this too?

@Inc this is cool but why are you just testing the models and not the products?

@Inc this is very cool stuff, gonna be looking at this benchmark everytime a new model drops!

@Inc This is awesome

@Inc Seems a lot like our reviewbench video 🤔

@Inc Amazing! Wondering why Gemini 3's latency is ~5x Claude/GPT-5

@Inc Really needed something like this!

@Inc "don't vibe check your vibe code" this is so cool! Love how you show the accuracy, cost/test, and time side by side. Makes it really easy to compare models.

@Inc 🍿

@Inc 🔥

@Inc Pretty cool! Curious to see which AIs can handle this vibe check and build a full web app

@Inc This is exciting!

@Inc this is huge congrats!

@Inc congrats!!

That’s honestly super cool. The whole “can it actually build a full app or is it just faking it” thing is a real gap right now. Curious how it scores with those models that love to hallucinate features. Do you have a public leaderboard yet, or is that coming? I can see this being super useful for devs trying to pick which AI model to trust with their bigger projects. Congrats on the feature too! 🚀

@Inc You can find the leaderboard here along with more details!

@Inc Oh nice, thanks for sharing the leaderboard link! Love seeing some transparency on how different models stack up. Gonna check it out now!

@Inc Very cool stuff @valsai. I wonder if the take away is that folks building coding tools ( Cursor, Copilot, Cognition, etc.) should not be wasting their time trying out the other models.

@Inc love this for evals!

@Inc Called out

@Inc This shows how great GPT-5.1 codex model is

@Inc Amazing work as always! :)

@Inc let’s gooooo

@Inc Love this! We’ve needed a benchmark that reflects actual end-to-end app creation. Vibe Code Bench is about to expose who’s really capable.

@Inc Finally a benchmark that matters! Model performance needs real-world context, not just buzz.

@Inc @theo interesting

@Inc Surprising that Gemini 3 Pro is still so far behind!

@Inc Finally, a test that actually measures output instead of hype. Curious how real the gap is.

@Inc game changer fr

@Inc really interesting results!

@Inc I love the video!!

@Inc huge if true

@Inc something about this feels disingenuous. and IMO there are too many confounders for this sort of “one-shot” testing to be useful.

@Inc Why wasn’t kimi k2 thinking benchmarked?

@Inc finally a bench that exposes ai's dev dreams. imagine it minting nfts that don't flop on gas fees

@Inc What about with agents? Codebuff, Kilo Code, Claude Code?