This is probably my most important post about DeepSeek V4 Flash. Do NOT judge this model after testing only one harness. Pi, Claude Code, and OpenCode produced dramatically different results—and OpenCode completely changed my verdict. You really need to read the details below.

This might be the most important lesson from all of my model benchmarks so far. I originally gave DeepSeek V4 Flash a negative assessment because I tested it through Pi. The generation was extremely poor, so naturally I assumed the model itself was the problem. But the more harnesses I tested, the stranger everything became. Pi produced a terrible result. Claude Code was somewhat better, but still not good enough—and it needed almost two hours. OpenCode produced an excellent result in roughly one hour. The map was far more complete, visually stronger, and potentially even better than GPT-5.6 Sol in certain areas. I also repeated the tests, and the results remained very similar. This does not appear to be a random lucky generation. What this tells me is incredibly important: Harness quality is model-dependent. A harness that works brilliantly for one model can perform terribly with another. I saw weak Qwen results through OpenCode, yet DeepSeek V4 Flash appears to work extremely well with it. I’ve noticed similar differences with Grok and other models too. So asking, “What is the best harness?” may be the wrong question. The correct question is: “What is the best harness for this specific model?” Please keep this in mind when looking at any benchmark—including mine. A single harness test may completely misrepresent what a model is actually capable of.

Try in pi using pi-codex-conversion extension. It handles the OpenAi Responses API way better which I believe v4 Flash has been trained on.

Vibe coding 3d simulations or games is pretty much the worst use of AI at this point in time. Models seem to be quite rightly focusing on UI and Backend development at this point in time. It seems obvious to me that 3d game coding requires a dedicated model.

opencode team is cooking

This is just normal variance between responses, likely nothing to do with harnesses. If you generate it 4 times with one harness you would still get very different results

no

why not

i repeated it multiple times, it is reproducible

oh-my-pi is the bestest ever.

do you also test oh my pi

单套评测翻车太常见了，得多跑几轮

@bridgemindai bro, please test multiple different harness next time.

I like these kinds of comparisons too, but you would actually need hundreds of generations to judge which one is really better.

What about hermes?

Opencode go and opencode zen (free dsv4) literally uses the same endpoint. AI is non-deterministic. That’s the reason.

Glad to see you did what i said, always benchmark models using the same harness, if you dont know how to use plan locked models (no api) on any harness, send me a DM and i can explain how, then you can use any plan on any harness.

opencode 🤯

Please try codex which is the recommended harness by DeepSeek 🙏

use oh-my-pi and test again

Lol the model literały told me it was engineered for Opencode

And each of these Chinese models have their own coding agent too. Eg reasonix is deepseek native and had enormous cache hit rate.

opencode go is not a harness but a subscription service... tf are you waffling

Also check ds4 agent made by @antirez

Wonder if it's because the DeepSeek team uses Opencode themselves?

Try Hermes! :)

Deepseek is made for agents so it should work the best in harnesses like hermes or openclaw

first screenshot was made with default model settings and default reasoning level in opencode. second screenshot was explicitly setting temp=1.0, top_p=0.95, reasoning=max (as recommended in the model card), still using opencode

Do NOT judge anything based on a single test. Especially as varying as output from an LLM.

Hit me with the prompt and I will try it on my homebrew harness

Just curious - with a non deterministic model - did you run each test multiple times and average them for each? Otherwise how do you know it wasn't just chance that one harness performed well and another didn't?

Firstly, you need to enable models hosted in China inside your OpenCode Go dashboard. Secondly, at that point, Flash Free and the version of Flash in Go are the same so add another harness.

Weird I get different result, it's very bad in Opencode desktop app but good in the cli

yeah ive tried it on multiple harnesses and major differences with several models, there needs to be a benchmark website

Is there any prompt available to reproduce this?

How are people doing these one prompt 4 models things? Does every harness get only 1 attempt ?

yes

Which is the best harness for qwen 3.6 35B-a3b. Am using opencode but still find Qwen’s output materially insufficient

can you share the prompt?