m4 mac mini cluster with 5 nodes running exo

M4 Mac Mini AI Cluster Uses EXO Labs with... Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis. The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon).show more

M4 Mac Mini AI cluster running Llama 3.3 70B... Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.show more

Running DeepSeek-V3 on M4 Mac Mini AI Cluster 671B... MoE model distributed across 8 M4 Pro 64GB Mac Minis. Apple Silicon with unified memory is a great fit for MoE.show more

Preparing to run the new Llama 3.3 70B on... my mac cluster. Downloading model shards onto 3 x M4 Pro Mac Mini and 1 x M3 Max MacBook Pro. AI cluster is connected by Gigabit ethernet switch with EXO Labsshow more

The tiny new Mac Mini - M4 and M4... Pro chips - 5 x 5 inch cube footprint - Headphone jack on the front - Starting at $599 with 16GB Memoryshow more

The new M4 Mac Mini is sooo mini!!

AGI at home Running DeepSeek R1 across my 7... M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)show more

How much faster is the new MacBook Pro for... AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.show more

Day 1: Benchmarks We ran 1000+ LLM benchmarks on... real consumer devices. Data includes single-device and multi-device clusters with Tokens-Per-Second (TPS) and Time-To-First-Token (TTFT). Setups tested: 3x M4 Mac Mini cluster, iPhone 15 + S24, RTX4090 & more.show more

Tested the new MacBook Pro M4 Pro vs. the... Mac mini M1 in LM Studio, running Llama 3.2 3B 4-bit MLX. Results: M1 (8-Core) ------> 25 tok/sec M4 Pro (14-Core) -> 104 tok/sec 🤯 – 4x faster!show more

Distributed training on M4 Mac Mini cluster We implemented... Google DeepMind DiLoCo on Apple Silicon to train large models with 100-1000x less bandwidth compared to DDP baseline. AI is entering a new era where a distributed network of consumer devices can train large models.show more

Success! 🍎🤖 I can now control Unitree robots on... a Mac Mini M4 running macOS with a simple tweak! The unitree_sdk2_python ( thread implementation isn’t essential—getting the DDS subscriber and publisher to work is enough. I replaced Unitree’s Linux thread logic with a simple Python while loop for sensor reading and robot control. Here’s a video of me controlling a Go2 robot on macOS (M4 Mac Mini) using a Python script for low-level control and sensor reading.show more

Running Kimi K2.5 on my desk. Runs at 24... tok/sec with 2 x 512GB M3 Ultra Mac Studios connected with Thunderbolt 5 (RDMA) using EXO Labs / MLX backend. Yes, it can run clawdbot.show more

A startup in China packed roughly 1,000 Mac Mini... M4 computers into a single data center to run AI workloads without paying ongoing cloud fees. The base Mac Mini M4 starts at $599 and draws just 10 to 30 watts under load, a fraction of the 300 to 500 watts a traditional GPU server pulls.show more

⚓ Kubernetes Architecture & Pod Creation 🧵- K8s cluster... consists of two types of nodes: 1️⃣ Control nodes (Master) 2️⃣ Compute nodes (Worker)show more

Watch Max, our VP of Customer Success, transform his... workspace with Apple Mac mini — with it's compact design and powerful M4 chip, his ordinary desk is now a creative hub.💡How are you stepping up your workspace for 2025? #smallbusinesssaturday #shopsmallsaturdayshow more

Running the full GLM 4.7 (8-bit) on 2 x... 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallelshow more

Running DeepSeek R1 on my desk Uses EXO Labs... with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.show more

i have to return my mac mini i need... a studio ASAP mac mini with 48gb CANNOT i repeat CANNOT run LLAMA 3.3 70B 😭😭🤦‍♂️🤦‍♂️🤦‍♂️show more