What if you could induce models to be more concise via pretraining data curation?

Pascal, 1657: "I have made this letter longer than usual because I have not had the time to make it shorter." Pascal didn’t know it at the time, but he was talking about inference. We made VLMs 35× cheaper per correct answer, by teaching them to be shorter. Entirely via pretraining data curation.

The conventional approach to inference efficiency is to shrink the model: distillation, pruning, quantization, speculative decoding. All of it optimizes FLOPs per token while treating the token count as fixed.

But it isn't fixed. So we addressed it with pretraining data curation. Concision in the data, and the model answers in a fraction of the tokens, with the same @datologyai accuracy improvements you know and love. No RL, no decoding tricks.

The headline: our Datology 4B spends 0.41 TFLOPs per correct answer. Qwen3.5-4B spends 14.58. That's 35× more at near-identical accuracy. Where does 35× come from? Primarily length. Our 4B emits ~30 tokens per answer. Qwen3.5-4B emits ~1,284.

In dollars, self-hosted on H100: ~$1.34 vs ~$47 per million correct answers. A soda vs a tank of gas, for the same correct answers. Curation pays once, before the model ships, and collects on every generation after.

The obvious worry is that short means dumb. It's the reverse. Hold output length fixed and the curated model is +17.55pp more accurate than its uncurated twin, and the edge grows with scale: up to +21.2pp at 4B. Brevity with no quality tax.

A key nuance: generic verbosity buys NO accuracy anywhere. Padding is just padding. Reasoning-structured verbosity (Qwen's chain-of-thought) helps at smaller scales, but doesn’t do much at larger scales: it improves 4 of 8 capability groups at 2B, down to 1 of 8 at 4B.

Interestingly, the concise curated model and the reasoning model have different error sets, implying that curation does not simply compress reasoning into fewer tokens, and that curating for reasoning and curating for brevity are complementary levers.

There's a name for this concept in cognitive science: amortized inference. The brain pays for a hard computation once, caches a cheap reusable version, and stops re-deriving it. Curation does the same for a model; System 2 deliberation compiled into a System 1 reflex.

Inference efficiency is a tokens-per-correct problem, and brevity is the lever that targets it directly. Shrinking the model and test-time solutions aren't the only move!

Huge thanks to @sjoshi804 @HaoliYin @RishabhAdiga01 @haakonmongstad and @alvind319 for developing the stellar VLM curation that this built on, and Sid in particular for deep, thorough feedback, and to the whole @datologyai team for being wonderful.

Paper: Blog: Join us: We also love interns and longer-term research collaborators If you're paying for inference bills, the cheapest token is the one your model never had to emit.