Skip navigation
this is why you think open source models aren't good
0:00 / 0:59

this is why you think open source models aren't good

dax

@thdxr191K followers

Visit on X
1,463 likesDownloadOriginal post

288,338 views

this is why you think open source models aren't good on the left is the average provider, on the right is opencode zen left is slow, vomits out tool call parse failures, confuses itself and takes 5x longer to complete

47 Comments

Comments from the original post on X (Twitter)

  • Kristoph's profile picture

    @kristoph

    this is a common open coder issue; especially if you run theough groq. kimi behaves much better using their api or if you host it yourself

    29 likes

  • Darren Shepherd's profile picture

    @ibuildthecloud

    This reenforces that fact that I don't care what anyone says about model performance. There's so many layers on top that people don't understand and rightly or wrongly attribute to the model.

    22 likes

  • dubaplex's profile picture

    @dubaplex

    I am going to use opencode zen for my apps inference and there is nothing you can do to stop me

    18 likes

  • dax's profile picture

    @thdxr

    @realduba we WANT you to do this the more usage we get flowing through the cheaper rates we can get for all of us use it and abuse it

    67 likes

  • Jay's profile picture

    @jayair

    @realduba kinky

    18 likes

  • dubaplex's profile picture

    @dubaplex

    @thdxr 😏

    1 like

  • realSpaceKangaroo's profile picture

    @_SpaceKangaroo_

    what are openrouter and groq specifically doing wrong to cause this? I get latency but if the prompt, model, and settings are all the same how? The model itself must be different somehow if this post is real.

    9 likes

  • Yaya Soumah's profile picture

    @YayaSoumah

    So do you agree now that implementation and vertical integration matters? A few weeks ago you was basically saying that models behave all the same in each application and that there was no such thing as model optimization

    9 likes

  • dax's profile picture

    @thdxr

    huh - this is a video of the exact same application side by side opencode zen is a provider, it'll be good in every application you use it with assuming they didn't fuck something up

    19 likes

  • Yaya Soumah's profile picture

    @YayaSoumah

    Then let me reframe, if the model is the same, why Opencode zed version behave much much better?

    3 likes

  • dax's profile picture

    @thdxr

    the idea here is the model is not the same when 2 providers claim to say "we provide kimi k2" you think you're getting the same product but they can serve it in ways that are good and in ways that are bad we just did the work to try out all the different ways to host it and only use the good ones

    59 likes

  • Yaya Soumah's profile picture

    @YayaSoumah

    Love this! Keep up!

    2 likes

  • Berend de Boer's profile picture

    @berenddeboer

    Yeah, this was a significant change, in the past I found opencode frustrating as I tried all these new models, but they were basically unusable. That's just fixed now.

    8 likes

  • Karim C's profile picture

    @BrandGrowthOS

    Provider infrastructure makes all the difference. Seen this with Claude too - same model, wildly different performance depending on who's serving it. The tooling layer matters more than people think.

    8 likes

  • Adam Pippert's profile picture

    @AdamPippert

    It’s almost as if the model is not the final product you receive from the API… Good job with zen, it’s clear you care about the final presentation and not just the model itself.

    7 likes

  • Jay's profile picture

    @jayair

    This is actually kinda crazy

    7 likes

  • Oliver's profile picture

    @olvrgln

    genuinely curious what's going on here. what's the secret sauce, is it literally just default parameters like temperature, etc?

    6 likes

  • burb ▲'s profile picture

    @molerburb

    i honestly think u should make the list public, so the bad providers are forced to correct it. though i also understand it's hard work and you may wanna keep it close to capture value

    5 likes

  • dax's profile picture

    @thdxr

    nah we will make it public (it's already visible in our github repo) - we just haven't been loud about it yet because we just launched this service and things may shift around

    13 likes

  • burb ▲'s profile picture

    @molerburb

    very nice to hear that!! appreciate it a lot!