GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push... up the frontier specifically for local hardware. Better kernels, inference tricks like spec decoding, quantizing models, model routing. 2. Usability / UX. To use Local AI today, you need to install CLIs, select a model, a quantization, a harness that works best with that model, and you need to configure your inference engine. It's not very accessible. More work needs to be done on making this seamless, so the user doesn't even know the AI is running locally.show more

How can I get an exo hoodie?

We're hosting our first ever in-person event soon in SF. We'll have merch there.

Oh wow, lucky for whoever goes!

can I get early access?

100%… also spot on with the vLLM part

EXO is doing an amazing work to push local AI to the masses. Particularly if you run heterogeneous deployments. With Spark Arena’s DGX Spark specific OSS projects is quite easy to run the latest vLLM, SGLang, llama.cpp, Atlas … versions on a single or multiple sparks using and also LLM recipes shared by real DGX Spark users around the world. In fact, just ask your agent to do it for you.

Yes. When you buy a computer, you don't need to configure drivers to get audio output. We need personal compute clusters anyone can buy off the shelf and start generating tokens within minutes.

The user shouldn't even know it's running locally is the right bar for usability honestly

Make your local machine work fully before you pay for cloud. That’s the hard part and these guys are making it more accessible maybe even easier.

@exolabs Hey I need this mad happy esque exo hoodie immediately, please and thank you

You need to get GLM-5.2 quality on single gpus if you really want to get local mass adopted

Enjoyed this. Excited to see what you have up next. I’m very pleased with @exolabs, Pi, GLM 5.2, and multiple memory systems on pre hooks with post hooks to memory for growth. I’m confident this is early days.. but most of my life I’m early and clunky… then someone comes in and streamlines all of it.. but that discovery and error curve is how I have always been able to stay investing in the right vehicles. Thanks to @exolabs and all the builders helping us break new ground.

100%, most people don't care if it's local or cloud, they care if it opens instantly and just works, the infrastructure should be invisible

I need this!!

@exolabs How many Mac Studio do I need to get GLM running with 5-10 tps on a local machine?

@exolabs Correct. OSS AI will run local and on the cloud. We are addressing the later with ease of use addressed out of the box.

W/ NVFP4 & MTP 5 Im getting 140 tok/s generation on the full glm-5.2 with the least used expert at 3 bit to make it fit. Uncompressed it would be 1.5 tb instead of 327 while losing only 1 point on GPQA Diamond.

What hardware

4 x rtx pro 6000

Strongly agree — clusters are a dead end for mass adoption. Axis 1 is already paying off via architecture: a 35B-A3B MoE (4-bit) does ~84 tok/s in 22GB on a single 64GB Mac — no cluster needed. Axis 2 is the real gap — I live the model/quant/harness config hell daily.

You nailed the two axes but missed the third: reliability. Mass adoption needs models that don't hallucinate your grandma's medical dosage. Intelligence and speed mean nothing if normies can't trust the output without verification. Who's working on that trust layer?

Agreed, but all I care about is hosting Z GLM 5.2 at home.

Local doesn't die on the frontier — it dies at "pick a quant and a harness." Ollama got closest by hiding it all behind one pull. Endgame: the app never says "local," it just runs. Frontier is table stakes; invisibility is the moat.

@exolabs Who is working on UX for local AI right now?

This form needs a few more tokens :)

Lack of vision really gimps the model

The ecosystem of opensource currently is fragmented and makes it laborious if not difficult to start building local inferencing infrastructure, once opensource nails this we can be better option for those who have access to hardware

Right

decode speed's the wall for local, not throughput. but benchmarks still chase inference latency. what's the real blocker in your deployments?

Would love access to local dot ai. Running gpt-oss-120b (76 tok/s), Qwen3-30B, and Gemma 4 12B via Ollama on a Mac Studio M3 Ultra, served over Tailscale to the rest of my machines. Claude Code daily in the terminal. RE developer using AI for market intelligence — moving more inference on-device.