Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Running DeepSeek-V3 on M4 Mac Mini AI Cluster 671B MoE model distributed across 8 M4 Pro 64GB Mac Minis. Apple Silicon with unified memory is a great fit for MoE.

exolabs's profile picture

EXO Labs

51,087 subscribers

719,293 görüntüleme • 1 yıl önce •via X (Twitter)

EğitimBilim & TeknolojiOyun
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

M4 Mac Mini AI Cluster Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis. The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon).
0:19

M4 Mac Mini AI Cluster Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis. The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon).

Alex Cheema

3,517,483 görüntüleme • 1 yıl önce

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)
0:14

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)

Alex Cheema

1,935,054 görüntüleme • 1 yıl önce

Distributed training on M4 Mac Mini cluster We implemented Google DeepMind DiLoCo on Apple Silicon to train large models with 100-1000x less bandwidth compared to DDP baseline. AI is entering a new era where a distributed network of consumer devices can train large models.
0:16

Distributed training on M4 Mac Mini cluster We implemented Google DeepMind DiLoCo on Apple Silicon to train large models with 100-1000x less bandwidth compared to DDP baseline. AI is entering a new era where a distributed network of consumer devices can train large models.

EXO Labs

347,697 görüntüleme • 1 yıl önce

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.
0:14

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.

Alex Cheema

992,735 görüntüleme • 1 yıl önce

The tiny new Mac Mini - M4 and M4 Pro chips - 5 x 5 inch cube footprint - Headphone jack on the front - Starting at $599 with 16GB Memory
0:13

The tiny new Mac Mini - M4 and M4 Pro chips - 5 x 5 inch cube footprint - Headphone jack on the front - Starting at $599 with 16GB Memory

Marques Brownlee

3,546,552 görüntüleme • 1 yıl önce

Got continuous batching working with SSMs in mlx-lm. Here's four OpenCode agents simultaneously running Nvidia's Nemotron Nano on 64GB M4 Max. This is a nice model for smaller machines since it's MoE + hybrid attention (small cache).
0:18

Got continuous batching working with SSMs in mlx-lm. Here's four OpenCode agents simultaneously running Nvidia's Nemotron Nano on 64GB M4 Max. This is a nice model for smaller machines since it's MoE + hybrid attention (small cache).

Awni Hannun

35,078 görüntüleme • 7 ay önce

M4 Mac Mini AI cluster running Llama 3.3 70B Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.
0:11

M4 Mac Mini AI cluster running Llama 3.3 70B Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.

Alex Cheema

73,090 görüntüleme • 1 yıl önce

MARCUS CHEN STACKED 30 MAC MINIS INTO AN AI SERVER FARM. ONE $599 MAC MINI REPLACES YOUR $200/MONTH CLAUDE CODE BILL WITH $3 IN ELECTRICITY two months ago a developer posted his claude code bill on reddit. $170 in 10 days. someone replied "i bought a mac mini m4. haven't paid anthropic since." apple stores ran out of mac minis the same week the m4 chip has 120 gb/s memory bandwidth and unified memory architecture. cpu and gpu share one pool so the model loads once and both read from it. a $599 mac mini runs ai faster than a $1,500 windows pc with a discrete gpu since january 2026 ollama supports the anthropic messages api format. claude code connects directly to your local mac mini with one environment variable. same interface, zero api costs, $0 per request a heavy developer pays $459 a month across claude code max, chatgpt pro, gemini, cursor and copilot. that's $5,508 a year. the mac mini pays off in 3 months and runs on $3 in electricity after that uber rolled out claude code to 5,000 engineers and burned through their $3.4 billion 2026 ai budget in 4 months. the people who own the hardware in 2026 are going to look very far ahead in 2028 bookmark this and read the article below
0:16

MARCUS CHEN STACKED 30 MAC MINIS INTO AN AI SERVER FARM. ONE $599 MAC MINI REPLACES YOUR $200/MONTH CLAUDE CODE BILL WITH $3 IN ELECTRICITY two months ago a developer posted his claude code bill on reddit. $170 in 10 days. someone replied "i bought a mac mini m4. haven't paid anthropic since." apple stores ran out of mac minis the same week the m4 chip has 120 gb/s memory bandwidth and unified memory architecture. cpu and gpu share one pool so the model loads once and both read from it. a $599 mac mini runs ai faster than a $1,500 windows pc with a discrete gpu since january 2026 ollama supports the anthropic messages api format. claude code connects directly to your local mac mini with one environment variable. same interface, zero api costs, $0 per request a heavy developer pays $459 a month across claude code max, chatgpt pro, gemini, cursor and copilot. that's $5,508 a year. the mac mini pays off in 3 months and runs on $3 in electricity after that uber rolled out claude code to 5,000 engineers and burned through their $3.4 billion 2026 ai budget in 4 months. the people who own the hardware in 2026 are going to look very far ahead in 2028 bookmark this and read the article below

starmex

357,559 görüntüleme • 2 ay önce

Success! 🍎🤖 I can now control Unitree robots on a Mac Mini M4 running macOS with a simple tweak! The unitree_sdk2_python ( thread implementation isn’t essential—getting the DDS subscriber and publisher to work is enough. I replaced Unitree’s Linux thread logic with a simple Python while loop for sensor reading and robot control. Here’s a video of me controlling a Go2 robot on macOS (M4 Mac Mini) using a Python script for low-level control and sensor reading.
0:11

Success! 🍎🤖 I can now control Unitree robots on a Mac Mini M4 running macOS with a simple tweak! The unitree_sdk2_python ( thread implementation isn’t essential—getting the DDS subscriber and publisher to work is enough. I replaced Unitree’s Linux thread logic with a simple Python while loop for sensor reading and robot control. Here’s a video of me controlling a Go2 robot on macOS (M4 Mac Mini) using a Python script for low-level control and sensor reading.

Tairan He

57,603 görüntüleme • 1 yıl önce

Watch Max, our VP of Customer Success, transform his workspace with Apple Mac mini — with it's compact design and powerful M4 chip, his ordinary desk is now a creative hub.💡How are you stepping up your workspace for 2025? #smallbusinesssaturday #shopsmallsaturday
0:19

Watch Max, our VP of Customer Success, transform his workspace with Apple Mac mini — with it's compact design and powerful M4 chip, his ordinary desk is now a creative hub.💡How are you stepping up your workspace for 2025? #smallbusinesssaturday #shopsmallsaturday

Sock Club

80,163 görüntüleme • 1 yıl önce

THIS $599 MAC MINI CAN DRIVE THREE DISPLAYS NATIVELY - THIS SETUP TOOK "MORE SCREEN SPACE" SO FAR IT ENDED WITH BINOCULARS. the M4 Mac mini shrinks a full desktop into a 5 x 5-inch box. the base model launched with: 10-core CPU 10-core GPU 16GB unified memory support for two 6K displays plus one 5K display the M4 Pro version can drive three 6K displays at 60Hz, pushing more than 60 million pixels at once. no DisplayLink software is required for the supported three-screen configuration. the video turns that spec into a monitor wall, then delivers the punchline when the creator reaches for binoculars. honest line: Apple guarantees up to three native displays. the clip does not prove every screen in the room is connected to this one Mac mini. the tiny computer solved the screen limit. the next bottleneck is finding the window you opened. bookmark & watch today ↓
0:15

THIS $599 MAC MINI CAN DRIVE THREE DISPLAYS NATIVELY - THIS SETUP TOOK "MORE SCREEN SPACE" SO FAR IT ENDED WITH BINOCULARS. the M4 Mac mini shrinks a full desktop into a 5 x 5-inch box. the base model launched with: 10-core CPU 10-core GPU 16GB unified memory support for two 6K displays plus one 5K display the M4 Pro version can drive three 6K displays at 60Hz, pushing more than 60 million pixels at once. no DisplayLink software is required for the supported three-screen configuration. the video turns that spec into a monitor wall, then delivers the punchline when the creator reaches for binoculars. honest line: Apple guarantees up to three native displays. the clip does not prove every screen in the room is connected to this one Mac mini. the tiny computer solved the screen limit. the next bottleneck is finding the window you opened. bookmark & watch today ↓

Grimmer

16,290 görüntüleme • 7 gün önce

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.
0:15

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

Alex Cheema

529,690 görüntüleme • 1 yıl önce

NVIDIA just dropped Nemotron-3-Nano:4b — a tiny 2.8GB model. Guess whose hardware runs it the fastest? - RTX 4090: 226 tok/s - RTX 3090: 187 tok/s - Mac Studio M2 Ultra: 86 tok/s - Mac Mini M4: 25 tok/s Home court advantage is real. Also trying a new layout with live performance charts. Lmk what you think!
0:13

NVIDIA just dropped Nemotron-3-Nano:4b — a tiny 2.8GB model. Guess whose hardware runs it the fastest? - RTX 4090: 226 tok/s - RTX 3090: 187 tok/s - Mac Studio M2 Ultra: 86 tok/s - Mac Mini M4: 25 tok/s Home court advantage is real. Also trying a new layout with live performance charts. Lmk what you think!

stevibe

127,495 görüntüleme • 5 ay önce

Want full control over the OpenClaw/Hermes Agent on your Mac Mini? Looking to build a Mac Mini Cluster with remote physical power control? Try #NanoKVM-Go! A single USB-C connection is all you need for complete remote access and Fingerbot power control!👇
0:11

Want full control over the OpenClaw/Hermes Agent on your Mac Mini? Looking to build a Mac Mini Cluster with remote physical power control? Try #NanoKVM-Go! A single USB-C connection is all you need for complete remote access and Fingerbot power control!👇

Sipeed

22,443 görüntüleme • 2 ay önce

A HARDWARE MODDER BOLTED A BATTERY PACK ONTO A $599 MAC MINI M4 AND TURNED A DESK SERVER INTO A BACKPACK AI RIG THAT RUNS FOR EIGHT HOURS WITHOUT A WALL OUTLET he posts a video of the mac mini with a slim battery module clamped to the side, USB-C plugged into the back. no power brick, no outlet, just the 10 to 30W draw of the M4 chip pulling from a 20,000mAh bank. a full work day on one charge. this is not a portable laptop trick. it is a desk-class AI server that fits in a sling bag the same $599 mac mini i recommended in last week's article as the silent default, now untethered. pair it with a mobile hotspot or a Starlink mini and you have a fully off-grid LLM stack that answers questions in a forest, on a plane, in a power outage. the model lives in the box, the battery powers the box, that is the whole stack this is not a gadget. it is the first time the words "AI server" and "fits in your bag" belong in the same sentence
0:19

A HARDWARE MODDER BOLTED A BATTERY PACK ONTO A $599 MAC MINI M4 AND TURNED A DESK SERVER INTO A BACKPACK AI RIG THAT RUNS FOR EIGHT HOURS WITHOUT A WALL OUTLET he posts a video of the mac mini with a slim battery module clamped to the side, USB-C plugged into the back. no power brick, no outlet, just the 10 to 30W draw of the M4 chip pulling from a 20,000mAh bank. a full work day on one charge. this is not a portable laptop trick. it is a desk-class AI server that fits in a sling bag the same $599 mac mini i recommended in last week's article as the silent default, now untethered. pair it with a mobile hotspot or a Starlink mini and you have a fully off-grid LLM stack that answers questions in a forest, on a plane, in a power outage. the model lives in the box, the battery powers the box, that is the whole stack this is not a gadget. it is the first time the words "AI server" and "fits in your bag" belong in the same sentence

shmidt

31,619 görüntüleme • 1 ay önce

THIS SHELF OF MAC MINIS REPLACES $4,080 A YEAR IN AI SUBSCRIPTIONS 00:02 the camera pans across a shelf of stacked Mac minis and the trick is obvious: that silent little farm runs the models you rent every month most people pay 7 companies for AI and use 3 of the tools. they forget the rest on the credit card and call it a stack the Mac mini M4 ends that. one shared memory pool means a $599 box runs 7B and 8B models faster than Windows machines that cost twice as much ollama pull, one command. open webui in one docker line. point Claude Code at localhost and it just works it draws 10 to 30 watts, sits silent next to a router, and runs 24/7 for $3 a month in power it pays back a $20 ChatGPT Plus sub in 3 months, then saves you $4,000 a year while the frontier still rents you compute every month you wait is another $340 gone for compute that fits on a shelf
0:10

THIS SHELF OF MAC MINIS REPLACES $4,080 A YEAR IN AI SUBSCRIPTIONS 00:02 the camera pans across a shelf of stacked Mac minis and the trick is obvious: that silent little farm runs the models you rent every month most people pay 7 companies for AI and use 3 of the tools. they forget the rest on the credit card and call it a stack the Mac mini M4 ends that. one shared memory pool means a $599 box runs 7B and 8B models faster than Windows machines that cost twice as much ollama pull, one command. open webui in one docker line. point Claude Code at localhost and it just works it draws 10 to 30 watts, sits silent next to a router, and runs 24/7 for $3 a month in power it pays back a $20 ChatGPT Plus sub in 3 months, then saves you $4,000 a year while the frontier still rents you compute every month you wait is another $340 gone for compute that fits on a shelf

Fokki

12,933 görüntüleme • 1 ay önce

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants
0:11

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants

Adrien Grondin

46,205 görüntüleme • 10 ay önce

This guy from China makes $12,500 a month walking around crypto conferences with a backpack and a Mac Mini inside it. The setup is a Mac Mini running OpenClaw on a CUKTECH 10 Mini battery. Every other creator at these events waits until they're back at the hotel to process and upload. By then the alpha is gone and the thread is late. His portable AI catches keynote audio on the spot, extracts what matters and posts while everyone else is still in the room listening.
0:10

This guy from China makes $12,500 a month walking around crypto conferences with a backpack and a Mac Mini inside it. The setup is a Mac Mini running OpenClaw on a CUKTECH 10 Mini battery. Every other creator at these events waits until they're back at the hotel to process and upload. By then the alpha is gone and the thread is late. His portable AI catches keynote audio on the spot, extracts what matters and posts while everyone else is still in the room listening.

0xMarioNawfal

69,013 görüntüleme • 1 ay önce

LayerAI Works with DeepSeek to Supercharge the LayerAI Product Suite 🌐 LayerAI integrates DeepSeek by leveraging several of its features and capabilities, focusing on areas where DeepSeek's strengths align with LayerAI's objectives. 👉 Learn more about our work with DeepSeek: - Model Deployment: DeepSeek-V3 & Coder-V2 for superior code generation, debugging, and language tasks. - DeepSeek-Coder-V2: Coding in 338 languages with a 128K context for complex structures. - Efficient Inference: Enhanced performance with DeepSeek's MLA & MoE tech. - IDE Assist: Real-time coding help, error catch, and auto-complete. - GitHub Sync: Better collab with seamless version control and reviews. This integration elevates our AI tools, making coding, learning, and teamwork smarter. More:
0:13

LayerAI Works with DeepSeek to Supercharge the LayerAI Product Suite 🌐 LayerAI integrates DeepSeek by leveraging several of its features and capabilities, focusing on areas where DeepSeek's strengths align with LayerAI's objectives. 👉 Learn more about our work with DeepSeek: - Model Deployment: DeepSeek-V3 & Coder-V2 for superior code generation, debugging, and language tasks. - DeepSeek-Coder-V2: Coding in 338 languages with a 128K context for complex structures. - Efficient Inference: Enhanced performance with DeepSeek's MLA & MoE tech. - IDE Assist: Real-time coding help, error catch, and auto-complete. - GitHub Sync: Better collab with seamless version control and reviews. This integration elevates our AI tools, making coding, learning, and teamwork smarter. More:

LayerAI | AI2Earn

47,905 görüntüleme • 1 yıl önce