Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

m4 mac mini cluster with 5 nodes running exo

digitalix's profile picture

Alex Ziskind

22,433 subscribers

362,753 görüntüleme • 1 yıl önce •via X (Twitter)

EğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

M4 Mac Mini AI Cluster Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis. The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon).
0:19

M4 Mac Mini AI Cluster Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis. The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon).

Alex Cheema

3,516,469 görüntüleme • 1 yıl önce

M4 Mac Mini AI cluster running Llama 3.3 70B Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.
0:11

M4 Mac Mini AI cluster running Llama 3.3 70B Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.

Alex Cheema

73,090 görüntüleme • 1 yıl önce

Running DeepSeek-V3 on M4 Mac Mini AI Cluster 671B MoE model distributed across 8 M4 Pro 64GB Mac Minis. Apple Silicon with unified memory is a great fit for MoE.
0:18

Running DeepSeek-V3 on M4 Mac Mini AI Cluster 671B MoE model distributed across 8 M4 Pro 64GB Mac Minis. Apple Silicon with unified memory is a great fit for MoE.

EXO Labs

719,293 görüntüleme • 1 yıl önce

Preparing to run the new Llama 3.3 70B on my mac cluster. Downloading model shards onto 3 x M4 Pro Mac Mini and 1 x M3 Max MacBook Pro. AI cluster is connected by Gigabit ethernet switch with EXO Labs
0:18

Preparing to run the new Llama 3.3 70B on my mac cluster. Downloading model shards onto 3 x M4 Pro Mac Mini and 1 x M3 Max MacBook Pro. AI cluster is connected by Gigabit ethernet switch with EXO Labs

Alex Cheema

181,917 görüntüleme • 1 yıl önce

The new M4 Mac Mini is sooo mini!!
0:10

The new M4 Mac Mini is sooo mini!!

iJustine

363,248 görüntüleme • 1 yıl önce

The tiny new Mac Mini - M4 and M4 Pro chips - 5 x 5 inch cube footprint - Headphone jack on the front - Starting at $599 with 16GB Memory
0:13

The tiny new Mac Mini - M4 and M4 Pro chips - 5 x 5 inch cube footprint - Headphone jack on the front - Starting at $599 with 16GB Memory

Marques Brownlee

3,545,527 görüntüleme • 1 yıl önce

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)
0:14

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)

Alex Cheema

1,934,842 görüntüleme • 1 yıl önce

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.
0:15

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

Alex Cheema

529,673 görüntüleme • 1 yıl önce

Day 1: Benchmarks We ran 1000+ LLM benchmarks on real consumer devices. Data includes single-device and multi-device clusters with Tokens-Per-Second (TPS) and Time-To-First-Token (TTFT). Setups tested: 3x M4 Mac Mini cluster, iPhone 15 + S24, RTX4090 & more.
0:18

Day 1: Benchmarks We ran 1000+ LLM benchmarks on real consumer devices. Data includes single-device and multi-device clusters with Tokens-Per-Second (TPS) and Time-To-First-Token (TTFT). Setups tested: 3x M4 Mac Mini cluster, iPhone 15 + S24, RTX4090 & more.

EXO Labs

254,968 görüntüleme • 1 yıl önce

Tested the new MacBook Pro M4 Pro vs. the Mac mini M1 in LM Studio, running Llama 3.2 3B 4-bit MLX. Results: M1 (8-Core) ------> 25 tok/sec M4 Pro (14-Core) -> 104 tok/sec 🤯 – 4x faster!
0:10

Tested the new MacBook Pro M4 Pro vs. the Mac mini M1 in LM Studio, running Llama 3.2 3B 4-bit MLX. Results: M1 (8-Core) ------> 25 tok/sec M4 Pro (14-Core) -> 104 tok/sec 🤯 – 4x faster!

01000010

111,446 görüntüleme • 1 yıl önce

Distributed training on M4 Mac Mini cluster We implemented Google DeepMind DiLoCo on Apple Silicon to train large models with 100-1000x less bandwidth compared to DDP baseline. AI is entering a new era where a distributed network of consumer devices can train large models.
0:16

Distributed training on M4 Mac Mini cluster We implemented Google DeepMind DiLoCo on Apple Silicon to train large models with 100-1000x less bandwidth compared to DDP baseline. AI is entering a new era where a distributed network of consumer devices can train large models.

EXO Labs

347,697 görüntüleme • 1 yıl önce

Success! 🍎🤖 I can now control Unitree robots on a Mac Mini M4 running macOS with a simple tweak! The unitree_sdk2_python ( thread implementation isn’t essential—getting the DDS subscriber and publisher to work is enough. I replaced Unitree’s Linux thread logic with a simple Python while loop for sensor reading and robot control. Here’s a video of me controlling a Go2 robot on macOS (M4 Mac Mini) using a Python script for low-level control and sensor reading.
0:11

Success! 🍎🤖 I can now control Unitree robots on a Mac Mini M4 running macOS with a simple tweak! The unitree_sdk2_python ( thread implementation isn’t essential—getting the DDS subscriber and publisher to work is enough. I replaced Unitree’s Linux thread logic with a simple Python while loop for sensor reading and robot control. Here’s a video of me controlling a Go2 robot on macOS (M4 Mac Mini) using a Python script for low-level control and sensor reading.

Tairan He

57,603 görüntüleme • 1 yıl önce

Running Kimi K2.5 on my desk. Runs at 24 tok/sec with 2 x 512GB M3 Ultra Mac Studios connected with Thunderbolt 5 (RDMA) using EXO Labs / MLX backend. Yes, it can run clawdbot.
0:18

Running Kimi K2.5 on my desk. Runs at 24 tok/sec with 2 x 512GB M3 Ultra Mac Studios connected with Thunderbolt 5 (RDMA) using EXO Labs / MLX backend. Yes, it can run clawdbot.

Alex Cheema

3,048,931 görüntüleme • 5 ay önce

A startup in China packed roughly 1,000 Mac Mini M4 computers into a single data center to run AI workloads without paying ongoing cloud fees. The base Mac Mini M4 starts at $599 and draws just 10 to 30 watts under load, a fraction of the 300 to 500 watts a traditional GPU server pulls.
0:13

A startup in China packed roughly 1,000 Mac Mini M4 computers into a single data center to run AI workloads without paying ongoing cloud fees. The base Mac Mini M4 starts at $599 and draws just 10 to 30 watts under load, a fraction of the 300 to 500 watts a traditional GPU server pulls.

Massimo

1,609,936 görüntüleme • 18 saat önce

⚓ Kubernetes Architecture & Pod Creation 🧵- K8s cluster consists of two types of nodes: 1️⃣ Control nodes (Master) 2️⃣ Compute nodes (Worker)
0:15

⚓ Kubernetes Architecture & Pod Creation 🧵- K8s cluster consists of two types of nodes: 1️⃣ Control nodes (Master) 2️⃣ Compute nodes (Worker)

Ayaan 🐧

21,256 görüntüleme • 1 yıl önce

Watch Max, our VP of Customer Success, transform his workspace with Apple Mac mini — with it's compact design and powerful M4 chip, his ordinary desk is now a creative hub.💡How are you stepping up your workspace for 2025? #smallbusinesssaturday #shopsmallsaturday
0:19

Watch Max, our VP of Customer Success, transform his workspace with Apple Mac mini — with it's compact design and powerful M4 chip, his ordinary desk is now a creative hub.💡How are you stepping up your workspace for 2025? #smallbusinesssaturday #shopsmallsaturday

Sock Club

80,163 görüntüleme • 1 yıl önce

Running the full GLM 4.7 (8-bit) on 2 x 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallel
0:12

Running the full GLM 4.7 (8-bit) on 2 x 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallel

Alex Cheema

156,668 görüntüleme • 6 ay önce

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.
0:14

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.

Alex Cheema

992,246 görüntüleme • 1 yıl önce

i have to return my mac mini i need a studio ASAP mac mini with 48gb CANNOT i repeat CANNOT run LLAMA 3.3 70B 😭😭🤦‍♂️🤦‍♂️🤦‍♂️
0:14

i have to return my mac mini i need a studio ASAP mac mini with 48gb CANNOT i repeat CANNOT run LLAMA 3.3 70B 😭😭🤦‍♂️🤦‍♂️🤦‍♂️

ashen

55,203 görüntüleme • 4 ay önce