正在加载视频...
视频加载失败
GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push... show more
30,854 次观看 • 3 个月前 •via X (Twitter)
30 条评论
How can I get an exo hoodie?
We're hosting our first ever in-person event soon in SF. We'll have merch there.
Oh wow, lucky for whoever goes!
can I get early access?
100%… also spot on with the vLLM part
EXO is doing an amazing work to push local AI to the masses. Particularly if you run heterogeneous deployments. With Spark Arena’s DGX Spark specific OSS projects is quite easy to run the latest vLLM, SGLang, llama.cpp, Atlas … versions on a single or multiple sparks using and also LLM recipes shared by real DGX Spark users around the world. In fact, just ask your agent to do it for you.
Yes. When you buy a computer, you don't need to configure drivers to get audio output. We need personal compute clusters anyone can buy off the shelf and start generating tokens within minutes.
The user shouldn't even know it's running locally is the right bar for usability honestly
Make your local machine work fully before you pay for cloud. That’s the hard part and these guys are making it more accessible maybe even easier.
@exolabs Hey I need this mad happy esque exo hoodie immediately, please and thank you
You need to get GLM-5.2 quality on single gpus if you really want to get local mass adopted
Enjoyed this. Excited to see what you have up next. I’m very pleased with @exolabs, Pi, GLM 5.2, and multiple memory systems on pre hooks with post hooks to memory for growth. I’m confident this is early days.. but most of my life I’m early and clunky… then someone comes in and streamlines all of it.. but that discovery and error curve is how I have always been able to stay investing in the right vehicles. Thanks to @exolabs and all the builders helping us break new ground.
100%, most people don't care if it's local or cloud, they care if it opens instantly and just works, the infrastructure should be invisible
I need this!!
@exolabs How many Mac Studio do I need to get GLM running with 5-10 tps on a local machine?
@exolabs Correct. OSS AI will run local and on the cloud. We are addressing the later with ease of use addressed out of the box.
W/ NVFP4 & MTP 5 Im getting 140 tok/s generation on the full glm-5.2 with the least used expert at 3 bit to make it fit. Uncompressed it would be 1.5 tb instead of 327 while losing only 1 point on GPQA Diamond.
What hardware
4 x rtx pro 6000
Strongly agree — clusters are a dead end for mass adoption. Axis 1 is already paying off via architecture: a 35B-A3B MoE (4-bit) does ~84 tok/s in 22GB on a single 64GB Mac — no cluster needed. Axis 2 is the real gap — I live the model/quant/harness config hell daily.
You nailed the two axes but missed the third: reliability. Mass adoption needs models that don't hallucinate your grandma's medical dosage. Intelligence and speed mean nothing if normies can't trust the output without verification. Who's working on that trust layer?
Agreed, but all I care about is hosting Z GLM 5.2 at home.
Local doesn't die on the frontier — it dies at "pick a quant and a harness." Ollama got closest by hiding it all behind one pull. Endgame: the app never says "local," it just runs. Frontier is table stakes; invisibility is the moat.
@exolabs Who is working on UX for local AI right now?
This form needs a few more tokens :)
Lack of vision really gimps the model
The ecosystem of opensource currently is fragmented and makes it laborious if not difficult to start building local inferencing infrastructure, once opensource nails this we can be better option for those who have access to hardware
Right
decode speed's the wall for local, not throughput. but benchmarks still chase inference latency. what's the real blocker in your deployments?
Would love access to local dot ai. Running gpt-oss-120b (76 tok/s), Qwen3-30B, and Gemma 4 12B via Ollama on a Mac Studio M3 Ultra, served over Tailscale to the rest of my machines. Claude Code daily in the terminal. RE developer using AI for market intelligence — moving more inference on-device.
![$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."](https://image.24vids.com/tw-2054708780114538680/amplify_video_thumb/2054708528296902656/img/mn2rvrKGvrwkqgQB.jpg)
