A 3B model just cleared a puzzle that a 1.6 TRILLION param model couldn't. You've seen this benchmark before: my sliding-puzzle test. Same Kimi & DeepSeek runs as last time. The only new thing: I dropped VibeThinker-3B in for a side-by-side. > VibeThinker → 3B > DeepSeek V4 Flash... → 284B > Kimi K2.6 → 1T > DeepSeek V4 Pro → 1.6T Shuffle depths 5, 10, 12, 15, 18, 22. One wrong move scrambles the whole board, so it's pure long-chain reasoning. ✅ VibeThinker-3B: solved all six. Never lost the thread. ⚠️ The giants started cracking at depth 15: Flash, Pro, and even Kimi each blew a run, scrambling the board past the move cap. As VibeThinker was not trained for tool calling, I had it emit X and ran the move. Bigger generalist ≠ smarter.show more

hugging face:

Not trained for tool calling you say… Maybe we can fix that 😍

Correct me if I'm wrong, but I saw that VibeThinker doesn't support tool calls at all, which limits what it can do. So we have to copy and paste the results from the chat the old-fashioned way, right?

I instructed it to respond in the old-fashioned way, something like <step>X</step>.

So this tiny model holds up well? Unbelievable. I want to try it.

Yep it did, you can see the small area of the model's log in the video.

Can't wait for tool calls to new officially supported in this model. Local AI is looking more promising than ever before.

A 3B model holding the thread longer than models 500x its size says a lot about how much long chain reasoning actually depends on training, not just scale

Benchmaxx

Those are some good vibes!

Love these posts. The clearest, simplest comparison of a real world test rather than just bench numbers. Thank you. Unreal the performance on Vibe 3B vs these monsters!

This is like a gymnast vs powerlifter. The 1.6T model has the mass, but a 3B model that nails a puzzle just has better proprioception.

let give this model fable 5 reasoning

What would happen if the 1.6T model had a suite of a dozen 3b models like this to use as tools?

Very interesting... I think I may have to give it a shot! Have you used it for code or tool calling yet?

Not yet. Actually, it is not trained for tool-calling, so using it directly with existing harnesses might not yield great performance. But I think it is great for solving hard math related questions.

That's nice, but what is the utility? What would you use this model for?

I'm not using it alone, maybe for math reasoning questions in an agentic workflow.

I mean with the right data you can make 3B beat basically anything

Breaking: Anthropic caught cheating. Reddit: Claude says is DeepSeek reports user.

This tracks with what we see on focused coding subTasks - a specialized small model often beats a much larger generalist.

Am curious, are the agents controlling the browser directly? Or if there is a game engine that the agents can interact with through command line, and have the webapp render the steps? Or something else?

There's a CLI tool that comes with multiple tools for the LLMs to solve the puzzle, including checking the current board state, moving a puzzle, etc., and I replayed it on the web app.

Thank you for getting back! Would you mind pointing to the CLI tool?