Skip navigation
We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4…
0:00 / 0:51

We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4…

Cactus Compute

@cactuscompute4.6K followers

Visit on X
2,912 likesDownloadOriginal post

600,496 views

We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4 Flash. One set of weights, every depth from 2 to 20 layers a model of its own, 25-121M parameters at CQ2-bit, built on our Simple Attention Networks and running locally at up to 4k tokens/sec decode speed on a Raspberry Pi 5. Needle does not chat. Every turn is a function call: give it the tools your app exposes and it picks the right ones and fills every argument from what the user said, or hand it a schema and it returns a typed record. Ask for something no tool covers and you get an empty list, not a guess. That trade is lets 121M parameters trained on 360B tokens of structured data beat models 10x their size on mobile tool calls and match 2-3x bigger models on structured JSON extraction. It runs on mobiles, wearables, smart home devices, small robots and microcontrollers, with prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, the browser and WASI hosts. Try it in your browser:

51 Comments

Comments from the original post on X (Twitter)

  • Cactus Compute's profile picture

    @cactuscompute

    Needle 3 is a Simple Attention Network. 1) The dense feed-forward layers are gone: a Monarch Hadamard MLP with 25.6K parameters per layer takes their place. 2) The knowledge they would hold sits in an engram, hashed n-gram tables read by gather at zero arithmetic. 3) 70.8M of the 121M parameters live there, so the full model does the arithmetic of a 50M one: 100 MFLOPs per token against 296 for a transformer of the same shape.

    19 likes

  • Cactus Compute's profile picture

    @cactuscompute

    The intelligence ladder. Every depth from 2 to 20 layers of the same weights is a trained, deployable model: blocks 0 and 19 are always kept and the rest are added by bisection, so each subnetwork nests in the next. A watch, a Pi and a phone pick their size at deploy time with `needle build --layers n` and run on the same engine.

    11 likes

  • Cactus Compute's profile picture

    @cactuscompute

    On Mobile Actions, 961 phone commands scored on the exact call, the 20-layer model reaches 86.0 through the shipped 2-bit binary with the confidence gate on. LFM2.5 1.2B is at 82.4, Qwen3.5 0.8B at 76.0, FunctionGemma 270M at 65.1 and Apple's on-device model at 57.6, all at f16. DeepSeek V4 Flash through its API is at 88.4, the line in every chart.

    8 likes

  • Cactus Compute's profile picture

    @cactuscompute

    Every argument is a span of the request. The model writes a one-line derivation first ('living room' -> room; '30' -> brightness), then emits the call under a byte-level grammar compiled from your schema, so the JSON always parses and an enum cannot leave its set. Optional field with no evidence: omitted. Required field with no evidence: the call is withheld. "Except the bedroom": the bedroom call is dropped.

    7 likes

  • Cactus Compute's profile picture

    @cactuscompute

    Structured extraction is the same operation with the record as the only tool. Declare the shape, pass the text, get a typed object whose every field is a span of the passage; an optional field with no span comes back as None instead of a guess. An enum field turns the record into a classifier whose output cannot leave the set.

    7 likes

  • Cactus Compute's profile picture

    @cactuscompute

    The small depths lose accuracy on the general benchmarks and get it back when fine-tuned to one product's tools. On DroidCall every subnetwork gains 18 to 36 points, and from 4 layers (29M parameters) up the tuned subnetwork passes DeepSeek V4 Flash. LoRA on the frozen base, merged at export, from `pip install cactus-needle` on a Mac or PC; the 2-bit post-training behind the shipped model runs on the Cactus Platform.

    5 likes

  • Cactus Compute's profile picture

    @cactuscompute

    The whole model is one .cact file the engine maps into memory and reads in place: a 196-byte header carrying the architecture, a nameless tensor directory, and weights at 2.125 bits each. Groups of 128 weights are rotated by a Walsh-Hadamard matrix, split into an fp16 norm and a direction, and the direction is snapped to a 4-entry codebook; the kernel never expands them

    8 likes

  • Cactus Compute's profile picture

    @cactuscompute

    Thirteen platform folders, one engine under 1MB each, all loading the same weights: macOS, Linux on x86-64, ARM64, ARMv7, RISC-V and MIPS32, Windows x64 and ARM, Android, iOS, watchOS, tvOS, the browser as WebAssembly, and a WASI component. `needle build --platform linux-arm64 --layers 8 --out ./pi` puts the engine and the weights in a folder you copy over; inference never touches the network.

    7 likes

  • Cactus Compute's profile picture

    @cactuscompute

    Every response carries a calibrated confidence score: act above your threshold, show the call and ask below it, or escalate to a bigger model. Needle in front of a private DeepSeek V4 Flash deployment is the setup we run for enterprise work. Guides, the Python reference and the playground are at thoughts always appreciated.

    7 likes

  • Karen's profile picture

    @kar_moss_

    The pareto frontier is moving

    5 likes

  • Gordon Olson's profile picture

    @Gordon_SonofOl

    You all are amazing!! Keep up the great work!

    4 likes

  • Cactus Compute's profile picture

    @cactuscompute

    thanks :)

    2 likes

  • Nico's profile picture

    @itmos__

    Please repost this but without the Claudish copywriting. This looks very interesting and relevant to my work, and I almost scrolled away because I can’t bear more of the “every x is a y, no z”meandering. All those words to say tool calling and typed structured outputs.

    3 likes

  • Cactus Compute's profile picture

    @cactuscompute

    Happy its relevant! Only experts really understand phrases like "tool calling", business-style writing helps clarify for the public. But there are technical deep dives in the blog for users like yourself:

  • Tony Yamin's profile picture

    @Tonyyamin_

    bullish

    3 likes

  • Florian S's profile picture

    @airesearch12

    Dude this is the second Jev like tech revolution in 48h. Give me a break.

    3 likes

  • Cactus Compute's profile picture

    @cactuscompute

    Maybe we are all unto something :)

    4 likes

  • Florian S's profile picture

    @airesearch12

    I bet you are! Congrats on the release. I know the feeling.

    1 like

  • Momcilo Mrkaic's profile picture

    @momcilo02

    Can’t wait to test this

    3 likes

  • Joe Gibbons's profile picture

    @JZGibbons

    Can this be used as a classifier to route tasks to different models depending on complexity and capabilities of said models?

    2 likes