Skip navigation
How to read CursorBench πŸ”Š
0:00 / 3:25

How to read CursorBench πŸ”Š

matt palmer

@mattyp134K followers

Visit on X
940 likesDownloadOriginal post

64,336 views

How to read CursorBench πŸ”Š

31 Comments

Comments from the original post on X (Twitter)

  • matt palmer's profile picture

    @mattyp

    See the full benchmark & learn more here:

    25 likes

  • Jay's profile picture

    @JayAndAI

    This is really helpful, especially for a beginner like me, thanks Matt.

    3 likes

  • matt palmer's profile picture

    @mattyp

    glad I could help!

    1 like

  • Cali's profile picture

    @itscali_ai

    Would love CursorBench split by task type. The model that wins big refactors isn't always the one that nails a quick webhook fix or a UI tweak. A "best model for this kind of task" view would save a ton of trial and error.

    2 likes

  • Angelo Sobrinho's profile picture

    @AngeloLSobrinho

    Do you intend to release a Composer 3? Composer 2.5 has a great cost-benefit ratio compared to Grok 4.7. I miss a cheap model in the Ultra X.

    1 like

  • Vamsi Batchu's profile picture

    @vamsibatchuk

    @tri333 your explainers are just so good, Matt ! πŸ”₯

    1 like

  • bnistordev's profile picture

    @bnistordev

    this is quite simple to understand for everyone, i'll take it and use it for the next event.

    1 like

  • TheCozyDev's profile picture

    @The_CozyDev

    Amazing. This is super useful, thank you Matt πŸ’œ

    1 like

  • matt palmer's profile picture

    @mattyp

    thank YOU

    1 like

  • David Wong's profile picture

    @davidwong

    Super useful and exciting to see. Thanks, Matt!

    1 like

  • matt palmer's profile picture

    @mattyp

    Thanks david!

    1 like

  • Eero Vuojolahti's profile picture

    @Vuojolahti

    Putting that zero into the right corner is not the way man

    1 like

  • Xor's profile picture

    @XorDev

    My one critique of auto routing models is that the task may be technically simple (e.g. UI adjustments), but a larger model tends to understand the context better, resulting in a better layout, formatting, UX, etc. I often find auto underperforms

    1 like

  • markus's profile picture

    @markusglob

    cost should increase from left to right. c'mon bro , common sense!

  • P's profile picture

    @P7602604897993

    Is this coming to Grok bot? I would love for a cheaper model to wake my bots and also bots using the cheapest model to share a message or communicate with each other. I’m on ultra, and my usage doesn’t go past 2 days….

  • Palm Beach AI's profile picture

    @PalmBeachAI

    Now, add in prompt caching, because this can flip the script on costs.

  • loccointelligence's profile picture

    @loccoai

    I'd suggest modifying the axes labels as follows: x-axis: Average cost per task (lower is better) y-axis: Score (higher is better) Yes, yes, in hindsight, this is trivial but when I first look at charts, I want to understand the axes right away, and more importantly, I want to know right away what the best "measurements" are. Also, it helps A LOT when comparing too, imo!

  • matt palmer's profile picture

    @mattyp

    good feedback

    1 like

  • loccointelligence's profile picture

    @loccoai

    that's why they pay me the big money, Matt! 😎

  • Tim Thyne's profile picture

    @timthyne

    I missed everything because ADHD brain was fixated on the inverse x axis.