Introducing Magnitude: your actually local agent 100% private and offline. No token costs, no API keys. Open source. Today's agents are local. The model isn't. Every prompt, every file, every secret gets sent straight to Anthropic and OpenAI. Magnitude is built around local models and runs the whole stack... itself. The inference engine is part of the agent, so the models run inside it, right on your computer. It lives in your terminal, and setup is one command. Magnitude profiles your hardware and shows you which models fit, with the trade-offs between quality, speed, and memory. Pick one and start working. No painful config or server to babysit. Out of the box, it can use your shell, edit files, and run scripts. Add skills and it can work with Excel, PowerPoint, PDFs or Chrome. Use it for everyday work: - Analyze sensitive data - Manage private notes - Review code and logs - Search and organize files - Build docs or slides npm i -g @magnitudedev/cli GitHub:show more

Shout out to @huggingface for their model hub. @UnslothAI for many of the quants we use. @EffectTS_ as the backbone of our agent lib. @vercel for the great skills library. @NVIDIARTXSpark for the Sparks we test Linux on. And many more

Using a firewall I verified that the only outbound connections were to github and huggingface, so the engineering here seems solid

Thanks for keeping us honest :)

Think you'll add vllm support eventually for leet self hosters?

We’re going to add a standard chat completions endpoint for people that want to bring their own inference server

This is awesome, looking forward trying it out

Let’s go! Let me know what you think

Will do 😃

what size MBP do you recommend? what are the min specs?

Ideally you want Apple silicon so post-2020 Macs. No hard RAM min, but with 32 GB you can fit a fair amount of the more capable smaller models like Qwen 3.6 35-A3B or Gemma 4 models

@dexhorthy What could it do with 16gb?

@dexhorthy Depends on the chip, but assuming an M4/M5 MacBook, you could run Qwen 3.5 9B or Gemma 4 12B which are capable for smaller tasks. Going to be tough to fit anything bigger without using a pretty aggressive quant, which lowers model quality

Not actually bad on average specs laptops. It provides 15 tokens/second. One thing i should suggest is different themes because on my light theme terminal i can't see the output of model.

I have a DGX Spark, but using it from mac or other Linux mini pc. Either local or via tailscale. Any option for remote to local inference? That way i don’t need to rsync files when working on something locally

We’re working on this right now, should release today. Going to be a generic chat completions endpoint to enable both remote inference and bring your own inference server. Eventually we want to make remote inference a more first class experience, but that will take some time

Thank you!

离线这点挺戳，密钥满天飞真的怕了

This is incredible. I have an idea… does it support ACP or any plan to?

Yes we plan to add support for ACP so we can integrate with other platforms... feel free to DM

got to see this in person at the workOS event, very cool!

Oh hey! Loved OpenInspect too. Open source ftw 🤝

here becasue Pash sent me

Pash counter +1

Gotta get pewdiepie on this

Congrats on the launch

Thanks Amir! You should give it a spin

love yalls demo wednesday the workos event. does this support any local model then

Any model you download via HF! But Magnitude also profiles your hardware and recommends good local models based on your machine, so it's not necessary to research models ahead of time

awesome

Fully offline with no token costs is a huge selling point.

Will definitely try it out. Open source and offline . Good job dude .

The annoying part of local AI has always been the setup.

this is next level on privacy protection 🤩

We got your back 😎

Thank you for open sourcing! ❤️

Of course! I hope people find it valuable :)

The future is now

Is inference built on llama cpp?

Yep! We also plan to add MLX support if we detect Apple silicon

Love the privacy angle! What is your biggest question about truly local AI agents?

Is it the same harness from previous launch one or two months ago with ability to use local models or something different?

It's roughly the same harness but with a built-in Rust inference engine for running local models. We've also made some tweaks to the harness to make it work even better for local models. And there's no more cloud models

this is wonderful, thankyou Tom!

Hope you do something cool with it :)

Awesome! The project I started but did not follow through on.

Well it's open source so feel free to extend it as much as you want :)

Hi Tom, nice work, was thinking of something similar. Do you have any harness capabilities to compensate for smaller models shortcomings, like forgetting, loops, bad reasoning?

Yes! We have a tool call circuit breaker to prevent doom loops. We also have measures in place to prevent overthinking. Overall we are also more conservative with context to prevent overwhelming smaller models context windows

Excellent! Varying the model params slightly also helps with the ruts to jump lanes. I have a bunch more ideas and a way to package these as a single file executable package bundle, config, prompts, tuning, params, etc.

This is interesting, going to give it a spin! Nice work

Thank you! Let me know how you get on

Very cool!

Thank you! Open source ftw

Magnitude you say…

It looks nice !! and yeah local stack is starting to feel less optional

Qwen3-27B makes it possible

Local-first agents are the move. Which model backend — llama.cpp, ollama, something else?

It's our own Rust inference engine based on llama.cpp, but we plan to also add automatic support for MLX if we detect Apple silicon

Which harness do you use? Or did you build one as well?

It’s our own! Built to work well with local models but also just be a generally great harness. It’s all open source on GitHub

Nice! I’ll check it out

this is pretty sick, would love to run some experiments. sent you a DM.

lfg legends

Thanks boss 🤝

What’s actually different than running local model with open code or pi agents with llama.cpp or any other inference? Also is this harness tune and optimise based on different open source model? Based on their straight and weakness?