GLM 5.3 Flash vs Hybrid (GLM 5.3 Flash + Jev) on a turn-based strategy game (Halite 2) The hybrid approach is 13x faster, 56% of the pure GLM 5.3 Flash API cost, and gains a slight edge in performance. In the hybrid approach: -GLM 5.3 Flash handles strategic judgment... of which planets matter, who expands, who fights, when to re-plan. -Jev handles the ship-by-ship execution. GLM 5.3 Flash alone defeats Jev alone 82% of the time. This is obviously just a single example and some of the first things I've tried to make use of Jev as an augmentation to an LLM. Could potentially make even more gains in cost, speed, and performance, but using what I've figured out here, I am compelled to try some ideas with robotics next.show more

funny that the cheaper faster option also just wins more, usually you have to pick two of those three

yeah these are surprising results with very little effort. This is basically all my first attempts too, only improvement from here

But how do you do the context transfer between the planner and actor? Especially when dealing with one way doors - like reflex games might work, but what about states which require way more planning or reasoning before taking action? (This is the problem I have at browser use when trying to make it supefast with Jevlike models)

Is the ship-by-ship execution non-deterministic? I'm wondering if having execution in code would be even better?

banger

curious whether the speedup holds across fresh map sets, or if particular map layouts are doing most of the work. great clean split though

can you say more? There are 133 unique maps, hence the different seeds per map here. Obviously different games and different ways of being able to parallelize will change how much speed/cost gains you might see though!

Got it, I meant generalization across map families, not just seeds. If those 133 maps span the layouts you care about, that answers it. A held-out batch from unseen map generators would be the next stress test.

The strategist/executor split makes sense to me. 13x faster at 56% of the cost with a performance edge is a great result. I think this architecture fits autonomous robotics in aerospace really well 🤖 Excited to see where you take it next!

Cool, can't wait to test it myself

the strategist/executor split is the only way autonomous loops stay cheap and stable in production. tested this running an autonomous demographic media pipeline for brazil from outside: frontier models outline the macro narrative once, then small decision models + local python/ffmpeg handle the per-chart rendering and validation. letting big models micromanage execution burns money for zero added variance.

13x faster at 56% of the cost, with a slight edge, is the result that survives a review. Model on the strategic call, fast path on the rest. If that holds past one game, the hybrid stops being a hack and becomes the default loop.

Interesting how Jev alone beats GLM flash alone 82% of the time, but combining them somehow results in an improvement over GLM flash alone...

GLM beats Jev 82% of the time. I don't know why you flipped it here :P

You're right. I somehow managed to write that completely backwards (even though I was thinking about it right in my head). Main point being that it's interesting/puzzling how instead of dragging GLM down, Jev combined with GLM seems to create some sort of synergistic enhancement.

hybrid llm routing is becoming the obvious play - why pay for full inference when you can route to cheaper models for 80% of decisions.

Thanks for sharing this

Please bring more use cases and possibilities!!

@MohitKarekar : check this

Slight edge = ? (how slight? 5% better? 10%?) @Sentdex

The cost savings compound when you're running thousands of game iterations. 56% of API cost per game means 1.8x more experiments in the same budget.

The next benchmark is recovery after one bad Jev handoff. Average win rate hides the expensive failure mode: one cheap wrong branch poisoning every high-cost step after it.

What robotics tasks are you thinking about running?

Check this

13x faster is the structural part, but the slight performance edge is what I wouldn't bank: with different seeds per map, a hybrid that only edges GLM sits inside run-to-run noise unless the same maps get played both ways.

13x faster with the hybrid approach is impressive. Excited to see how it performs with robotics!

feels like some sort of polymarket betting automation is a perfect application for this combo

When it comes to decision-making and execution in turn-based strategy games like Halite2, it looks like LLMs can indeed make a huge difference when used correctly.

Awesome man! Btw you can use free inference on GLM 5.3 flash through @LatchLoopAI Been doing that last few weeks

Decoupling macro strategy from micro execution always felt like the right pattern, but seeing a 13x speedup is wild.

The 13x speedup is the interesting part, not the cost. Curious whether Jev is handling the per-turn loop while GLM only gets called on branch points, or if it's arbitrating every decision. That distinction usually decides whether the hybrid stays faster as game length grows.

13x speedup at 56% quality is usable if the cost per decision drops proportionally. Did you measure token cost difference between pure and hybrid?

13x speed, half the cost, and slightly better in perf. There was no downside to using it. Sorry if it was confusing :P

How much of the 13x speedup comes from running ship decisions in parallel, versus replacing individual GLM calls with Jev?

13x faster at half the cost is the trade i keep making in my own agents. most steps are easy calls that never needed the smartest model.

A 13x speedup at 56% of the API cost with a slight performance edge is a strong result for the hybrid approach. Turn-based strategy is a great testbed — would be interesting to see whether the division of labor (strategy to Flash, execution to Jev) holds up in real-time games.

The strategy vs ship-by-ship split makes the handoff really clear. Curious where you’d draw that boundary in robotics before an irreversible action.