Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

M4 Mac Mini AI Cluster Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis. The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon).

Alex Cheema

53,219 subscribers

3,517,633 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

M4 Mac Mini AI cluster running Llama 3.3 70B Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.

M4 Mac Mini AI cluster running Llama 3.3 70B Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.

Alex Cheema

73,090 views • 1 year ago

Running DeepSeek-V3 on M4 Mac Mini AI Cluster 671B MoE model distributed across 8 M4 Pro 64GB Mac Minis. Apple Silicon with unified memory is a great fit for MoE.

Running DeepSeek-V3 on M4 Mac Mini AI Cluster 671B MoE model distributed across 8 M4 Pro 64GB Mac Minis. Apple Silicon with unified memory is a great fit for MoE.

EXO Labs

719,293 views • 1 year ago

Preparing to run the new Llama 3.3 70B on my mac cluster. Downloading model shards onto 3 x M4 Pro Mac Mini and 1 x M3 Max MacBook Pro. AI cluster is connected by Gigabit ethernet switch with EXO Labs

Preparing to run the new Llama 3.3 70B on my mac cluster. Downloading model shards onto 3 x M4 Pro Mac Mini and 1 x M3 Max MacBook Pro. AI cluster is connected by Gigabit ethernet switch with EXO Labs

Alex Cheema

181,983 views • 1 year ago

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.

Alex Cheema

992,805 views • 1 year ago

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)

Alex Cheema

1,935,289 views • 1 year ago

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

Alex Cheema

529,690 views • 1 year ago

Day 1: Benchmarks We ran 1000+ LLM benchmarks on real consumer devices. Data includes single-device and multi-device clusters with Tokens-Per-Second (TPS) and Time-To-First-Token (TTFT). Setups tested: 3x M4 Mac Mini cluster, iPhone 15 + S24, RTX4090 & more.

Day 1: Benchmarks We ran 1000+ LLM benchmarks on real consumer devices. Data includes single-device and multi-device clusters with Tokens-Per-Second (TPS) and Time-To-First-Token (TTFT). Setups tested: 3x M4 Mac Mini cluster, iPhone 15 + S24, RTX4090 & more.

EXO Labs

255,162 views • 1 year ago

"It's on my todo to build my own stack of apple minis and run Llama on my own little cluster on my desk." - Garry Tan Quick AI homelab tutorial: - Install EXO Labs on each mini (open source, steps in README) - Make sure all the minis are on the same WiFi network or for faster/more reliable results, connected over Thunderbolt/Ethernet. Devices will auto-discover each other and auto-shard the LLMs. - Now you can chat to your cluster with your preferred LLM GUI (the tiny corp tinychat ships with exo)

"It's on my todo to build my own stack of apple minis and run Llama on my own little cluster on my desk." - Garry Tan Quick AI homelab tutorial: - Install EXO Labs on each mini (open source, steps in README) - Make sure all the minis are on the same WiFi network or for faster/more reliable results, connected over Thunderbolt/Ethernet. Devices will auto-discover each other and auto-shard the LLMs. - Now you can chat to your cluster with your preferred LLM GUI (the tiny corp tinychat ships with exo)

Alex Cheema - e/acc

243,248 views • 1 year ago

Distributed training on M4 Mac Mini cluster We implemented Google DeepMind DiLoCo on Apple Silicon to train large models with 100-1000x less bandwidth compared to DDP baseline. AI is entering a new era where a distributed network of consumer devices can train large models.

Distributed training on M4 Mac Mini cluster We implemented Google DeepMind DiLoCo on Apple Silicon to train large models with 100-1000x less bandwidth compared to DDP baseline. AI is entering a new era where a distributed network of consumer devices can train large models.

EXO Labs

347,745 views • 1 year ago

EXO 1.0 at Apple NeurIPS booth. 4 x 512GB M3 Ultra Mac Studios clustered all-to-all with RDMA over Thunderbolt 5. Runs DeepSeek v3.2 (8-bit) at 25 tok/sec with tensor parallelism.

EXO 1.0 at Apple NeurIPS booth. 4 x 512GB M3 Ultra Mac Studios clustered all-to-all with RDMA over Thunderbolt 5. Runs DeepSeek v3.2 (8-bit) at 25 tok/sec with tensor parallelism.

EXO Labs

186,329 views • 8 months ago

Success! 🍎🤖 I can now control Unitree robots on a Mac Mini M4 running macOS with a simple tweak! The unitree_sdk2_python ( thread implementation isn’t essential—getting the DDS subscriber and publisher to work is enough. I replaced Unitree’s Linux thread logic with a simple Python while loop for sensor reading and robot control. Here’s a video of me controlling a Go2 robot on macOS (M4 Mac Mini) using a Python script for low-level control and sensor reading.

Success! 🍎🤖 I can now control Unitree robots on a Mac Mini M4 running macOS with a simple tweak! The unitree_sdk2_python ( thread implementation isn’t essential—getting the DDS subscriber and publisher to work is enough. I replaced Unitree’s Linux thread logic with a simple Python while loop for sensor reading and robot control. Here’s a video of me controlling a Go2 robot on macOS (M4 Mac Mini) using a Python script for low-level control and sensor reading.

Tairan He

57,603 views • 1 year ago

MARCUS CHEN STACKED 30 MAC MINIS INTO AN AI SERVER FARM. ONE $599 MAC MINI REPLACES YOUR $200/MONTH CLAUDE CODE BILL WITH $3 IN ELECTRICITY two months ago a developer posted his claude code bill on reddit. $170 in 10 days. someone replied "i bought a mac mini m4. haven't paid anthropic since." apple stores ran out of mac minis the same week the m4 chip has 120 gb/s memory bandwidth and unified memory architecture. cpu and gpu share one pool so the model loads once and both read from it. a $599 mac mini runs ai faster than a $1,500 windows pc with a discrete gpu since january 2026 ollama supports the anthropic messages api format. claude code connects directly to your local mac mini with one environment variable. same interface, zero api costs, $0 per request a heavy developer pays $459 a month across claude code max, chatgpt pro, gemini, cursor and copilot. that's $5,508 a year. the mac mini pays off in 3 months and runs on $3 in electricity after that uber rolled out claude code to 5,000 engineers and burned through their $3.4 billion 2026 ai budget in 4 months. the people who own the hardware in 2026 are going to look very far ahead in 2028 bookmark this and read the article below

MARCUS CHEN STACKED 30 MAC MINIS INTO AN AI SERVER FARM. ONE $599 MAC MINI REPLACES YOUR $200/MONTH CLAUDE CODE BILL WITH $3 IN ELECTRICITY two months ago a developer posted his claude code bill on reddit. $170 in 10 days. someone replied "i bought a mac mini m4. haven't paid anthropic since." apple stores ran out of mac minis the same week the m4 chip has 120 gb/s memory bandwidth and unified memory architecture. cpu and gpu share one pool so the model loads once and both read from it. a $599 mac mini runs ai faster than a $1,500 windows pc with a discrete gpu since january 2026 ollama supports the anthropic messages api format. claude code connects directly to your local mac mini with one environment variable. same interface, zero api costs, $0 per request a heavy developer pays $459 a month across claude code max, chatgpt pro, gemini, cursor and copilot. that's $5,508 a year. the mac mini pays off in 3 months and runs on $3 in electricity after that uber rolled out claude code to 5,000 engineers and burned through their $3.4 billion 2026 ai budget in 4 months. the people who own the hardware in 2026 are going to look very far ahead in 2028 bookmark this and read the article below

starmex

357,679 views • 2 months ago

THIS $599 MAC MINI CAN DRIVE THREE DISPLAYS NATIVELY - THIS SETUP TOOK "MORE SCREEN SPACE" SO FAR IT ENDED WITH BINOCULARS. the M4 Mac mini shrinks a full desktop into a 5 x 5-inch box. the base model launched with: 10-core CPU 10-core GPU 16GB unified memory support for two 6K displays plus one 5K display the M4 Pro version can drive three 6K displays at 60Hz, pushing more than 60 million pixels at once. no DisplayLink software is required for the supported three-screen configuration. the video turns that spec into a monitor wall, then delivers the punchline when the creator reaches for binoculars. honest line: Apple guarantees up to three native displays. the clip does not prove every screen in the room is connected to this one Mac mini. the tiny computer solved the screen limit. the next bottleneck is finding the window you opened. bookmark & watch today ↓

THIS $599 MAC MINI CAN DRIVE THREE DISPLAYS NATIVELY - THIS SETUP TOOK "MORE SCREEN SPACE" SO FAR IT ENDED WITH BINOCULARS. the M4 Mac mini shrinks a full desktop into a 5 x 5-inch box. the base model launched with: 10-core CPU 10-core GPU 16GB unified memory support for two 6K displays plus one 5K display the M4 Pro version can drive three 6K displays at 60Hz, pushing more than 60 million pixels at once. no DisplayLink software is required for the supported three-screen configuration. the video turns that spec into a monitor wall, then delivers the punchline when the creator reaches for binoculars. honest line: Apple guarantees up to three native displays. the clip does not prove every screen in the room is connected to this one Mac mini. the tiny computer solved the screen limit. the next bottleneck is finding the window you opened. bookmark & watch today ↓

Grimmer

16,290 views • 16 days ago

THIS SHELF OF MAC MINIS REPLACES $4,080 A YEAR IN AI SUBSCRIPTIONS 00:02 the camera pans across a shelf of stacked Mac minis and the trick is obvious: that silent little farm runs the models you rent every month most people pay 7 companies for AI and use 3 of the tools. they forget the rest on the credit card and call it a stack the Mac mini M4 ends that. one shared memory pool means a $599 box runs 7B and 8B models faster than Windows machines that cost twice as much ollama pull, one command. open webui in one docker line. point Claude Code at localhost and it just works it draws 10 to 30 watts, sits silent next to a router, and runs 24/7 for $3 a month in power it pays back a $20 ChatGPT Plus sub in 3 months, then saves you $4,000 a year while the frontier still rents you compute every month you wait is another $340 gone for compute that fits on a shelf

THIS SHELF OF MAC MINIS REPLACES $4,080 A YEAR IN AI SUBSCRIPTIONS 00:02 the camera pans across a shelf of stacked Mac minis and the trick is obvious: that silent little farm runs the models you rent every month most people pay 7 companies for AI and use 3 of the tools. they forget the rest on the credit card and call it a stack the Mac mini M4 ends that. one shared memory pool means a $599 box runs 7B and 8B models faster than Windows machines that cost twice as much ollama pull, one command. open webui in one docker line. point Claude Code at localhost and it just works it draws 10 to 30 watts, sits silent next to a router, and runs 24/7 for $3 a month in power it pays back a $20 ChatGPT Plus sub in 3 months, then saves you $4,000 a year while the frontier still rents you compute every month you wait is another $340 gone for compute that fits on a shelf

Fokki

12,933 views • 1 month ago

A 27-year-old guy from China bought 100 Mac minis and turned his apartment into a server hub Instead of furniture, the space is filled with neat racks where a hundred Mac minis hum like a living organism, processing gigabytes of data in a continuous stream Many small AI startups in Asia cannot afford to train models on NVIDIA H100 servers: it is insanely expensive He provides his Mac minis for running and fine-tuning small, highly specialized models. He takes on the dirty work of data labeling and quality assurance by using his network of agents that check each other's work > His initial investment was $59,900 > He was able to recoup his costs in 3 months Right now, his average income is $25,000–$30,000 per month He is not a genius: he simply found a niche with high demand and created an offer

A 27-year-old guy from China bought 100 Mac minis and turned his apartment into a server hub Instead of furniture, the space is filled with neat racks where a hundred Mac minis hum like a living organism, processing gigabytes of data in a continuous stream Many small AI startups in Asia cannot afford to train models on NVIDIA H100 servers: it is insanely expensive He provides his Mac minis for running and fine-tuning small, highly specialized models. He takes on the dirty work of data labeling and quality assurance by using his network of agents that check each other's work > His initial investment was $59,900 > He was able to recoup his costs in 3 months Right now, his average income is $25,000–$30,000 per month He is not a genius: he simply found a niche with high demand and created an offer

Bober_smart

553,644 views • 1 month ago

Inkling-small is out today! With SGLang, you can get 648 tok/s decode with DSpark (simulated acc len=4) and 288 tok/s w/o DSpark, under the same setup (8x NVIDIA AI B200, TP 8, NVFP4, bs=1). What makes this model different is the size. 276B total with 12B active is a sweet spot for RL, and both LoRA and full-parameter training become well within reach. Miles is ready and verified for multimodal RL on Inkling-small, so you can turn your multimodal data into real capability gains. At ~1/4 the size, Inkling-small matches the bigger version in capability and even wins on some benchmarks. Run Inkling-small with SGLang, and customize it with Miles.

Inkling-small is out today! With SGLang, you can get 648 tok/s decode with DSpark (simulated acc len=4) and 288 tok/s w/o DSpark, under the same setup (8x NVIDIA AI B200, TP 8, NVFP4, bs=1). What makes this model different is the size. 276B total with 12B active is a sweet spot for RL, and both LoRA and full-parameter training become well within reach. Miles is ready and verified for multimodal RL on Inkling-small, so you can turn your multimodal data into real capability gains. At ~1/4 the size, Inkling-small matches the bigger version in capability and even wins on some benchmarks. Run Inkling-small with SGLang, and customize it with Miles.

LMSYS Org

119,759 views • 26 days ago

Exploring diff visuals with Gen2. I'm amazed at how it takes a single block image to build neon streets, structures, and entire cityscapes. Anybody else toying with AI tonight⁉️ AI Art Workflow: 1. Crafted starting image in #Midjourney 2. Generated a 4-sec video with #Gen2 (no text prompt) 3. Extended first 4 secs to 18 secs 4. Repeated steps 2-3 twice, each time using the final frame from the previous 18-sec video to start the next 18-secs run 5. Merged all three 18-sec videos in #FinalCut Pro 6. Adjusted speed for fluidity 7. Final version published here is a 54-sec journey distilled into a 16-sec video with 🎶 #AIart #AIArtCommunity

Exploring diff visuals with Gen2. I'm amazed at how it takes a single block image to build neon streets, structures, and entire cityscapes. Anybody else toying with AI tonight⁉️ AI Art Workflow: 1. Crafted starting image in #Midjourney 2. Generated a 4-sec video with #Gen2 (no text prompt) 3. Extended first 4 secs to 18 secs 4. Repeated steps 2-3 twice, each time using the final frame from the previous 18-sec video to start the next 18-secs run 5. Merged all three 18-sec videos in #FinalCut Pro 6. Adjusted speed for fluidity 7. Final version published here is a 54-sec journey distilled into a 16-sec video with 🎶 #AIart #AIArtCommunity

Dave Villalva

47,946 views • 3 years ago

$We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: 🧠 LLMs & Reasoning → DeepSeek-V4-Flash-0731 (my king 👑): 304B MoE refresh, Terminal-Bench 2.1 jumps 61.8→82.7 over the preview, DeepSWE 7.3→54.4. Closes in on Opus-4.8 on Agents' Last Exam (25.2 vs 25.7). MIT. → Muse-Glimmer-30B, from Meta (they are back!!): their first open agentic model. ~29.6B dense + perception encoder, 131k+ context, built to run fully local, no cloud. Apache 2.0. → Liquid AI LFM2.5-2.6B: 2.69B params, 131k context, 220 tok/s on an M5 Max in under 2.5GB RAM. Competitive with models 4x larger on agentic tasks. → inclusionAI Ling-3.0-flash: 124B total, only 5.1B active, ~12% the size of their old 1T flagship Ring-2.6, matches it on key benchmarks. MIT. → inclusionAI Ling-3.0-tiny: 7.9B total, 1.3B active, 86-90 tok/s on an M4 Pro MacBook at ~8GB peak memory. MIT. → NVIDIA Nemotron-3.5-Lightning-30B-A3B: hybrid Mamba-2+MoE+Attention, up to 1M context, runs on a single H100 or DGX Spark, SWE-bench Verified 52.8. → deepgrove maple-preview: 20B-A1B ternary-weight reasoner, 218 tok/s on a Mac mini M4, 5.3GB checkpoint. MIT. → BigBang-v1 (endless-frontier): fine-tuned from Qwen3.6-35B-A3B via a self-evolving generator/critic synthetic-data loop. Lands aggregate performance between DeepSeek V4 Flash (284B) and V4 Pro (1.6T), at 35B. Apache 2.0. 🎬 Video → MiniMax-H3: 33B dense omni model, native stereo audio, up to 2K/15s. 3.6k+ likes already. → Minimax-H3-Turbo (lightx2v): Apache-2.0 turbo distillation of H3 for fast inference. → Lightricks LTX-2.5: image-to-video update, custom Gemma-4-12B text encoder, a markedly stronger distilled model. 🔊 Voice → NVIDIA NemotronLabs VoiceChat-11B: full-duplex speech-to-speech, ~450ms turn-taking, #2 on open VoiceBench, and the first open full-duplex model with live tool-calling mid-conversation. 🛡️ Safety → Mistral Shieldstral-1.0-3B: 3B multimodal guardrail that takes your safety policy as plain text instead of fixed categories. Beats LlamaGuard-4-12B and ShieldGemma-9B on HarmBench (99.4) and ToxicChat (84.1) at a fraction of the size. Apache 2.0.$

We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: 🧠 LLMs & Reasoning → DeepSeek-V4-Flash-0731 (my king 👑): 304B MoE refresh, Terminal-Bench 2.1 jumps 61.8→82.7 over the preview, DeepSWE 7.3→54.4. Closes in on Opus-4.8 on Agents' Last Exam (25.2 vs 25.7). MIT. → Muse-Glimmer-30B, from Meta (they are back!!): their first open agentic model. ~29.6B dense + perception encoder, 131k+ context, built to run fully local, no cloud. Apache 2.0. → Liquid AI LFM2.5-2.6B: 2.69B params, 131k context, 220 tok/s on an M5 Max in under 2.5GB RAM. Competitive with models 4x larger on agentic tasks. → inclusionAI Ling-3.0-flash: 124B total, only 5.1B active, ~12% the size of their old 1T flagship Ring-2.6, matches it on key benchmarks. MIT. → inclusionAI Ling-3.0-tiny: 7.9B total, 1.3B active, 86-90 tok/s on an M4 Pro MacBook at ~8GB peak memory. MIT. → NVIDIA Nemotron-3.5-Lightning-30B-A3B: hybrid Mamba-2+MoE+Attention, up to 1M context, runs on a single H100 or DGX Spark, SWE-bench Verified 52.8. → deepgrove maple-preview: 20B-A1B ternary-weight reasoner, 218 tok/s on a Mac mini M4, 5.3GB checkpoint. MIT. → BigBang-v1 (endless-frontier): fine-tuned from Qwen3.6-35B-A3B via a self-evolving generator/critic synthetic-data loop. Lands aggregate performance between DeepSeek V4 Flash (284B) and V4 Pro (1.6T), at 35B. Apache 2.0. 🎬 Video → MiniMax-H3: 33B dense omni model, native stereo audio, up to 2K/15s. 3.6k+ likes already. → Minimax-H3-Turbo (lightx2v): Apache-2.0 turbo distillation of H3 for fast inference. → Lightricks LTX-2.5: image-to-video update, custom Gemma-4-12B text encoder, a markedly stronger distilled model. 🔊 Voice → NVIDIA NemotronLabs VoiceChat-11B: full-duplex speech-to-speech, ~450ms turn-taking, #2 on open VoiceBench, and the first open full-duplex model with live tool-calling mid-conversation. 🛡️ Safety → Mistral Shieldstral-1.0-3B: 3B multimodal guardrail that takes your safety policy as plain text instead of fixed categories. Beats LlamaGuard-4-12B and ShieldGemma-9B on HarmBench (99.4) and ToxicChat (84.1) at a fraction of the size. Apache 2.0.

Victor M

53,929 views • 13 days ago

This guy bought a $400 Mac mini and walked into a coffee shop with $2,100 for installing an AI agent that never touches the internet. I had to rewatch this because the pitch is almost too simple. He shows up, plugs in the Mac mini, installs a local AI agent running entirely through Ollama, done in under an hour. Zero cloud dependency, no API bills, no monthly charges bleeding the client dry. The coffee shop owner gets a private AI system that keeps working even if the WiFi dies, and he walks out with $2,100 for an install that takes less time than a long lunch. The hardware cost him $400. One deployment and the machine's already paid for five times over. Then he locks in a monthly retainer for support, somewhere around $100-150, which means every client after the first is close to pure margin stacking on top of recurring revenue. Most people hear "AI business" and think they need to build a SaaS platform or learn to code or raise funding from someone. This skips all of that and goes straight to walking into coffee shops with a Mac mini under one arm. Not a company that scales to a billion dollars, obviously, but a solo operator clearing a few thousand a month from local installs while everyone else is still arguing about which LLM is best on Twitter.

This guy bought a $400 Mac mini and walked into a coffee shop with $2,100 for installing an AI agent that never touches the internet. I had to rewatch this because the pitch is almost too simple. He shows up, plugs in the Mac mini, installs a local AI agent running entirely through Ollama, done in under an hour. Zero cloud dependency, no API bills, no monthly charges bleeding the client dry. The coffee shop owner gets a private AI system that keeps working even if the WiFi dies, and he walks out with $2,100 for an install that takes less time than a long lunch. The hardware cost him $400. One deployment and the machine's already paid for five times over. Then he locks in a monthly retainer for support, somewhere around $100-150, which means every client after the first is close to pure margin stacking on top of recurring revenue. Most people hear "AI business" and think they need to build a SaaS platform or learn to code or raise funding from someone. This skips all of that and goes straight to walking into coffee shops with a Mac mini under one arm. Not a company that scales to a billion dollars, obviously, but a solo operator clearing a few thousand a month from local installs while everyone else is still arguing about which LLM is best on Twitter.

BrainRul

4,690,435 views • 25 days ago