We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4 Flash. One set of weights, every depth from 2 to 20 layers a model of its own, 25-121M parameters at CQ2-bit, built on our Simple Attention Networks and running locally at up to 4k... tokens/sec decode speed on a Raspberry Pi 5. Needle does not chat. Every turn is a function call: give it the tools your app exposes and it picks the right ones and fills every argument from what the user said, or hand it a schema and it returns a typed record. Ask for something no tool covers and you get an empty list, not a guess. That trade is lets 121M parameters trained on 360B tokens of structured data beat models 10x their size on mobile tool calls and match 2-3x bigger models on structured JSON extraction. It runs on mobiles, wearables, smart home devices, small robots and microcontrollers, with prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, the browser and WASI hosts. Try it in your browser:show more

Needle 3 is a Simple Attention Network. 1) The dense feed-forward layers are gone: a Monarch Hadamard MLP with 25.6K parameters per layer takes their place. 2) The knowledge they would hold sits in an engram, hashed n-gram tables read by gather at zero arithmetic. 3) 70.8M of the 121M parameters live there, so the full model does the arithmetic of a 50M one: 100 MFLOPs per token against 296 for a transformer of the same shape.

The intelligence ladder. Every depth from 2 to 20 layers of the same weights is a trained, deployable model: blocks 0 and 19 are always kept and the rest are added by bisection, so each subnetwork nests in the next. A watch, a Pi and a phone pick their size at deploy time with `needle build --layers n` and run on the same engine.

On Mobile Actions, 961 phone commands scored on the exact call, the 20-layer model reaches 86.0 through the shipped 2-bit binary with the confidence gate on. LFM2.5 1.2B is at 82.4, Qwen3.5 0.8B at 76.0, FunctionGemma 270M at 65.1 and Apple's on-device model at 57.6, all at f16. DeepSeek V4 Flash through its API is at 88.4, the line in every chart.

Every argument is a span of the request. The model writes a one-line derivation first ('living room' -> room; '30' -> brightness), then emits the call under a byte-level grammar compiled from your schema, so the JSON always parses and an enum cannot leave its set. Optional field with no evidence: omitted. Required field with no evidence: the call is withheld. "Except the bedroom": the bedroom call is dropped.

Structured extraction is the same operation with the record as the only tool. Declare the shape, pass the text, get a typed object whose every field is a span of the passage; an optional field with no span comes back as None instead of a guess. An enum field turns the record into a classifier whose output cannot leave the set.

The small depths lose accuracy on the general benchmarks and get it back when fine-tuned to one product's tools. On DroidCall every subnetwork gains 18 to 36 points, and from 4 layers (29M parameters) up the tuned subnetwork passes DeepSeek V4 Flash. LoRA on the frozen base, merged at export, from `pip install cactus-needle` on a Mac or PC; the 2-bit post-training behind the shipped model runs on the Cactus Platform.

The whole model is one .cact file the engine maps into memory and reads in place: a 196-byte header carrying the architecture, a nameless tensor directory, and weights at 2.125 bits each. Groups of 128 weights are rotated by a Walsh-Hadamard matrix, split into an fp16 norm and a direction, and the direction is snapped to a 4-entry codebook; the kernel never expands them

Thirteen platform folders, one engine under 1MB each, all loading the same weights: macOS, Linux on x86-64, ARM64, ARMv7, RISC-V and MIPS32, Windows x64 and ARM, Android, iOS, watchOS, tvOS, the browser as WebAssembly, and a WASI component. `needle build --platform linux-arm64 --layers 8 --out ./pi` puts the engine and the weights in a folder you copy over; inference never touches the network.

Every response carries a calibrated confidence score: act above your threshold, show the call and ask below it, or escalate to a bigger model. Needle in front of a private DeepSeek V4 Flash deployment is the setup we run for enterprise work. Guides, the Python reference and the playground are at thoughts always appreciated.

The pareto frontier is moving

You all are amazing!! Keep up the great work!

thanks :)

Please repost this but without the Claudish copywriting. This looks very interesting and relevant to my work, and I almost scrolled away because I can’t bear more of the “every x is a y, no z”meandering. All those words to say tool calling and typed structured outputs.

Happy its relevant! Only experts really understand phrases like "tool calling", business-style writing helps clarify for the public. But there are technical deep dives in the blog for users like yourself:

bullish

Dude this is the second Jev like tech revolution in 48h. Give me a break.

Maybe we are all unto something :)

I bet you are! Congrats on the release. I know the feeling.

Can’t wait to test this

Can this be used as a classifier to route tasks to different models depending on complexity and capabilities of said models?

@Scrambledo so, the context size is 512 excluding the provided tool list and system prompt.

wtf?

🫶 thank you Cactus 🌵 love this series of models mi pi 5 is gonna be so happy 😛❤️

let us know how it goes!

@cactuscompute thanks for the model, we took it on a run on our endpoint. Hope it helps.

thanks for sharing :)

"does not chat" as a feature is elite. empty list beats a confident wrong tool call

I was hoping for an update, now really looking forward to testing this. Is recommended tools count still less than five?

The context size was doubled, so you can scale up to 10, here's a blog on recommendation:

That's great, thanks.

4k tokens/sec on a Pi 5 is wild. That decode speed changes the game for local robotics control loops.

your model just caused a nuclear fusion event in my bedroom

Now you can automate your home to cook intruders when away...literally.

Cool!

fake N scam! deepseek v4 is LLM gen ai. i have try your model via demo link not even close to deepsek like WTF?

dude so many crazy drops today wtffff

Needle 3: > can match DeepSeek V4 Flash > runs locally at up to 4k tokens/sec decode speed on a Raspberry Pi 5

Nice!

I can do things with this… time to break back out the ESP32…

Worth splitting the 121M: 70.8M of it is engram tables, hashed 2- and 3-gram memory at 18,432 slots across layers 3, 7, 11, 15 and 19, read by gather. Only ~50M does arithmetic, and their Figure 1 puts that at over 2x fewer MFLOPs per token than a same-shape transformer.

The MFLOP performance is a result of the Hadamard MLP, we explain it here:

the sneaky win is deploying only the depth each action surface needs. agent fleets waste a stupid amount of money routing trivial tool calls through the same-sized brain as hard decisions.

Which license covers the weights? The page only says source is on GitHub

Apache 2.0, its free to use :)

This is not a model, why compare with LLM. These are tool calls strutured as JSON.

deepseek's baseline on that chart is 60.5

You’re my new hero if this works as well as you claim

No pressure...

Absolutely love this. Beautiful work!

We are moving towards the world that I want: AI makes decision instead of yapping for ages.

🤝