Video wird geladen...
Video konnte nicht geladen werden
You already gave the LLM the answer. So why is it writing it out from scratch? Introducing a new AI model architecture: Constrained Decoding Models (CDMs) LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable. CDMs are purpose-built for 1 task each.... show more
1,410,953 Aufrufe • vor 1 Tag •via X (Twitter)
67 Kommentare
im not even done testing yesterday's model that a new class just spawned wtf
shit moves fast these days - now you can bench them!
I think that the next AI phase may be about figuring out which things don't need that model at all means we don't need the smartest model ever for every single task more specifications for each task
100% the future is multi-model and it's not going to be general purpose models across the stack
Using a frontier model to classify or tag at scale always felt like overkill. Great to see a model family built for exactly these tasks. Also very excited to try out the multimodal decision model.
multi-label is supported on scaledown classify so you can tag 1 input (text + image) with 20+ tags in 1 pass not just select 1 from a list
Woah 40x cheaper on input than Gemini 4 Argon 👀
"Output is free (yes, even for summarize)" brooo wow-
🧑🍳👩🍳👨🍳
when you say one model one job, is classify meant to replace the gpt routing call or sit in front of it?
yea classify can be used a router model. but it is not tuned for that task specifically. we have a router and tool calling model in the works. will be out soon!
this is truly great, huge practical low-cost solution. pulling a due date out of a 300-page PDF shouldn’t need a model that writes.
appreciate it!! why wait for generation when you already gave the answwer
Cool to avoid the unnecessary text and an answer straight away.
no need to waste time/money on reasoning tokens!
wasnt expecting Jev to get mogged so fast
😶🌫️
Hiring a giant model to sort tickets was always overkill. Glad someone built the right tool for it.
hire an intern for intern tasks
I like the framing here. LLMs write, but apps often just need classify, extract, or summarize.
couldn't agree more! why have 1 model that's ok when you can have 3 that are really good
if an invoice has two totals or a missing field, does Extract flag it or take a guess?
you can select the number of entities you want extracted so can do more than 1 and for missing fields extract will not return anything - never guesses main thing is you can set the confidence threshold for the entities as an input that is used for extract
Half of protecting data is knowing what’s in it. This makes that a lot cheaper. Awesome launch!
100% extract is great for PII redaction and other security/compliance use cases!
lemme give this a read Very similar to how decision models work, as far as I can understand, but seems like an interesting approach
similar but we've expanded past just decisions to data extraction and summary selection
Speed + cost numbers like this don’t come around often.
not everyday but we're happy to help people scale down their AI cost and latency!
Awesome, how should we call this? structured outputs?
the models are called constrained decoding models! for outputs i'd say extract = structured outputs but summarize is summaries and classify is decisions / labels
The funniest part is we’ve spent years optimizing prompts to make LLMs output valid JSON instead of questioning why we needed a generative model for that task in the first place 😭
lolll we did that too - we were just working with the tools we had happy to report no longer need to do that anymore!
31x size of Jev!! I can already think of a few solo SaaS startup ideas to build with it
love to hear that! can't see what you build also don't forget about images
really cool, invoice processing, support routing from screenshots and checking whether an agent is stuck in a loop
just the tip of the iceberg of use cases!
Faster results at a lower cost would help many teams.
which teams do you think would benefit the most?
I think for AI startups building agents that need fast classification, structured data extraction n summarization at scale.
Picking one task per model is the part I like most!! When a model only classifies, it stops wandering off into essays. So clean :)
appreciate it! 1 model per 1 task is the future
@Mr_Derivatives LLM’s are like wives and girlfriends 😉
@Mr_Derivatives i would love this to be elaborate on. im sure it's a good joke just missing the punchline
does Extract still return the exact source text and its position when the input is a scanned PDF image?
how does it handle ambiguous classifications?
you can add those details in the class labels! we have an agent that helps with prompt tuning to cover those edge cases:
Ngl not every ai workflow needs a massive LLM burning tokens just to classify or extract a few things
you might be onto something!
the 200x speed claim definitely gets attention.
love to hear it! test it out and lmk how it goes
you already gave the llm the answer and then paid it to type that same answer back slower.
seems counterintuitive no? i agree - hence scaledown
needed something like this
amazing! get your 50M tokens for free here
a new architecture for a faster answer. very interesting.
have to build from the ground up to get these 200x and 467x faster results
absolutely. there's a lot of room to innovate based on use cases. specialized models and architectures are going to be a big part of that.
CDM concept finally fixes production latency
why go slow when you can go fast🏎️
Time to run it up 🔥🔥🔥
🏃🏃♂️🏃♀️
i’m tired of paying for expensive llm that doesn’t exactly this trying this with the free tokens now
the rewrite step wastes tokens and latency when the model could just return the cached answer. biggest win would be skipping regeneration entirely for exact matches in context.
that's the base premise of CDMs
CDMs could make production AI workflows faster, cheaper, and more reliable.
100% that's the mission
