Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions. CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks. With lightweight... fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks. We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed. Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size. 📄 Blog: 💻 Code: 🗣️ Discord: 🤗 Data & Models: More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵show more

🧵(1) Model Architecture CLM first trains a state encoder and an action encoder on a large-scale dataset with a contrastive objective, so that each state is pulled toward the ground-truth action and pushed away from all others. The two encoders then serve directly as a zero-shot action classifier.

🧵(2) Data Recipe We release CLM-8B, which is pre-trained on 60M Nemotron Q&A pairs, mid-trained on 30M synthetic hard negatives, and post-trained on 1M agentic trajectories.

🧵(3) Scaling Laws for Verification We find that the test InfoNCE loss scales as a power law with training compute, dataset size, projection-head size, and encoder size. These dimensions must be scaled jointly to achieve optimal performance. Notably, scaling the encoder size yields the strongest gains.

🧵(4) Latency vs. Jev and Constrained Decoding The key difference between CLM and Jev is that Jev only supports state caching, whereas CLM’s dual-encoder architecture allows state and action embeddings to be cached independently. This is particularly useful in applications such as tool calling, computer use, and games, where the action space is predefined and remains fixed. By caching the action embeddings, CLM substantially reduces inference cost, with the efficiency gains becoming even larger as the number and length of candidate actions grow. At ~1K candidates, CLM is 13× faster than Jev ⚡

🧵(5) Zero-Shot Evaluation Across computer-use, gaming, and tool-calling tasks, CLM-8B performs on par with Jev while running up to 9× faster. The speedups are most pronounced when the number of candidates is large (e.g., WikiRacing) or when actions can be reused frequently across states (e.g., T-Rex Game). CLM-35B, with improved generalization and even greater speedups, will be released early next month.

🧵(6) Agentic Benchmarks We find that Jev fails to serve as a verifier for long-horizon tasks, performing below the random-selection (Pass@1) baseline. In contrast, with lightweight fine-tuning, CLM achieves SOTA performance on challenging agentic benchmarks, including DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%), while delivering 4–6× faster inference than Jev.

🧵(7) Dino Run Demo

🧵(8) Super Mario Demo

🧵(9) WikiRacing Demo

CLM comes with an interactive playground on GitHub:

CLM-8B is part of our scaling ladder, where we train models across multiple scales to establish scaling laws and predict performance at larger scales. A multimodal CLM-35B is now in training with more data, compute, and parameters. Stay tuned for the release early next month 🚀

Joint work with @hangoo_kang @TarunSures41845 @JonSaadFalcon @drmapavone @Azaliamirh and @HazyResearch

the disaggregation is the part builders should steal: a loop scores the same state against many candidate actions, so the state embedding caches perfectly and each candidate collapses to a cheap dot product. that's the economics that makes per-turn decisions affordable.

Exactly!

Very nice, but didn't Jev just come out? Could you explain how you guys did this work so quickly?

I actually explored this for robotics last year, and CLM grew directly out of that work a few months ago. Feel free to check out my earlier post for more details on how CLM could be applied to robotics:

Contrastive learning is back again!

More bookmarks than likes. That's how you know someone cooked here

adding to the benchmark

impressive stuff!

Vision support?

We’ll be releasing CLM-35B next month with vision support. Stay tuned for the release 👀

caching state and action embeddings separately is the clever bit. huge for tool routing

Finally, this is what I call real research

Do you want me to include it in JevBench?

Wait, so this just crushes Jev, and is free, and runs locally with 8 billion parameters? It fits in under 8gb? What's the context window size? What's the catch?! Don't toy with me here, this sounds so good..

We gotta get it numbers on decision bench and jev bench

@grok explain exactly what this is, and how can I use it, for what. Compare with Jev from typesafeAi

This model is up next on the benchmark chopping block. I hope it pushes the frontier!

is this OSS on hugging face? how does this perform on language recognition applications like "what was the users intention - here's waht they said - here's your options" ?

9x faster inference with open weights and published scaling laws is real progress. I'm curious what the compute footprint per agent task looks like vs a dense autoregressive model. Efficiency gains like this matter more than another benchmark point.

Really cool work! We’re building JevEmbed to turn embeddings into decisions: Choice, Score, and Noul judgments from existing embedding models, without text generation. Curious how CLMs would perform as a backbone.

how can a 8B model get these numbers?? DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). and no body is surprised??

if it's really 9× faster, that could change the game for real-time applications

The independent action cache is the appealing part for me: a changing state doesn't force you to re-encode the same menu of actions. For tool routing, have you tried a two-stage version—cache tool selection, then generate the arguments? That seems a useful middle ground between four fixed game moves and entirely fresh action candidates.

the useful leap here is treating state and action as separate objects instead of collapsing everything into a single prompt. cached embeddings can preserve what the run learned while the action set stays auditable. that is where faster inference becomes a continuity primitive, not just a benchmark win.

10x faster... that's insane

The claim to check is variance under shift. Contrastive objectives are only as good as the negatives they were trained against.

Very interesting work Jacky ! Bravo to the team 👏👏 Just out of curiosity: Was the idea originally inspired by robotics?

seems really useful for game agents

@jackyk02 missing piece next to big coding models agent loops burn tokens on tiny which-action-next calls. a contrastive system-1 that maps state to action without paragraphs feels closer to a real controller curious if 9x holds in a multi-tool harness with long history

is this a Fair and Honest measure of latency if you can tell based on the details shared? i.e. testing latency of an API to a model sitting on hardware right next to you vs. going thru all the standard DNS pipes etc of the web to get a response from Jev's API X miles away (if it's this scenario, it would not be a fair comparison)

InfoNCE indeed was essential, also multi-modality. Am working on a different project and approach and appreciate a lot your creative and beatiful results. Congratulations Jacky 😎🥂🍾🚀

@arankomatsuzaki This is sick!

Is CLM-8B open weights like Kev, or API only? Would love to see it on the same out-of-domain test Kev posted.

@grok please generate the comprehensive accurate prompt for my ai agent to set this up on my mac if possible if not will set it up on the cloud. Thank you

this is so good