Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

mlx distributed uses mpi for machine to machine communication, and this is how you can handle uneven distribution of model layers. Thanks Awni Hannun for the help

digitalix's profile picture

Alex Ziskind

9,650 subscribers

31,265 görüntüleme • 1 yıl önce •via X (Twitter)

OyunBilim & TeknolojiEğitim
LIVE

Anya Rossi• Live Now

Private livecam show

11 Yorum

abdou profil fotoğrafı
abdou1 yıl önce

@awnihannun v interesting, would love to know how it was ordering the machines before you specified it in the rank file. would be funny if it was just sorting the host names 😅

Alex Ziskind profil fotoğrafı
Alex Ziskind1 yıl önce

@awnihannun the mysteries of mpi.

Rainmaker profil fotoğrafı
Rainmaker2 yıl önce

Can Machine Learning beat the market? Check out this post on my free Substack where I share code and commentary for an XGBoost model and a Random Forest model that both deliver powerful performances.

Mason James.3.2-12b_q4_K_S profil fotoğrafı
Mason James.3.2-12b_q4_K_S1 yıl önce

@awnihannun Repo Prompt! ❤️ cc @pvncher

Art Friedman profil fotoğrafı
Art Friedman1 yıl önce

@awnihannun thanks for sharing! Great question for you, m3 ultra 512 vs multiple Mac mini m4 with the same budget, which option is better for LLM context?

Joe Burnett profil fotoğrafı
Joe Burnett1 yıl önce

@awnihannun Awesome!

Mid Season Group profil fotoğrafı
Mid Season Group1 yıl önce

@awnihannun @Apple engineering should be ashamed.

Leonard Sullivan profil fotoğrafı
Leonard Sullivan1 yıl önce

@awnihannun I love these mini racks.

Alex Ziskind profil fotoğrafı
Alex Ziskind1 yıl önce

@awnihannun yep It’s a DeskPi- @Mandy_Li888

moskstraumen profil fotoğrafı
moskstraumen1 yıl önce

@awnihannun Tell us more about the technical aspects, if you want!

CodeBreaker profil fotoğrafı
CodeBreaker1 yıl önce

@awnihannun love to see it

Benzer Videolar

DAAUUUMMMM! Deep Seek R1 - 4bit on a single Mac Studio 512gb. 18.26 Tokens per second with MLX. Took over a minute to load the model but I sped that up. Generation was great! thanks Awni Hannun mlx is the future.
1:30

DAAUUUMMMM! Deep Seek R1 - 4bit on a single Mac Studio 512gb. 18.26 Tokens per second with MLX. Took over a minute to load the model but I sped that up. Generation was great! thanks Awni Hannun mlx is the future.

Austin Vance

172,817 görüntüleme • 1 yıl önce

Llama 3 running locally on iPhone with MLX Built by exo intern team Mohamed Baioumy h/t Awni Hannun MLX & Prince Canuma for the port
0:24

Llama 3 running locally on iPhone with MLX Built by exo intern team Mohamed Baioumy h/t Awni Hannun MLX & Prince Canuma for the port

Alex Cheema

295,820 görüntüleme • 2 yıl önce

Your Mac is about to run inference like a datacenter. Coming soon to MLX-Swift: Continuous batching: the fastest way to handle multiple inference streams locally. It starts with regular inference and seamlessly upgrades to batched mode when new requests arrive. The best of both worlds. Based on the work of and Awni Hannun
0:25

Your Mac is about to run inference like a datacenter. Coming soon to MLX-Swift: Continuous batching: the fastest way to handle multiple inference streams locally. It starts with regular inference and seamlessly upgrades to batched mode when new requests arrive. The best of both worlds. Based on the work of and Awni Hannun

Ronald Mannak

60,570 görüntüleme • 7 ay önce

oMLX is working really well as single machine inference engine for coding agents! Caching is managed perfectly (it can use a ton of disk space, be aware!) and oQ quantization delivers great results. Behind the scenes it uses the standard MLX building blocks (75% created by Prince Canuma 🙏): - mlx-lm - mlx-vlm - mlx-embeddings - mlx-audio I tested Qwen3.6-35B-A3B-oQ6 on M5 Max with two pi instances and it was fast and furious and leveraging cache like crazy as you can see in the video. Let me try to create some oQ versions (2,4,6?) of MiniMax M2.7 now and then I'll pass to distributed inference. I must win! 💪
3:57

oMLX is working really well as single machine inference engine for coding agents! Caching is managed perfectly (it can use a ton of disk space, be aware!) and oQ quantization delivers great results. Behind the scenes it uses the standard MLX building blocks (75% created by Prince Canuma 🙏): - mlx-lm - mlx-vlm - mlx-embeddings - mlx-audio I tested Qwen3.6-35B-A3B-oQ6 on M5 Max with two pi instances and it was fast and furious and leveraging cache like crazy as you can see in the video. Let me try to create some oQ versions (2,4,6?) of MiniMax M2.7 now and then I'll pass to distributed inference. I must win! 💪

Ivan Fioravanti ᯅ

19,268 görüntüleme • 2 ay önce

For the chest and lapels, these layers can be attached to each other using a single-needle roll-padding machine, such as you see here. This is what you'll typically see on factory-made suits (this is a Strobel KA-ED machine). Happens both on the low- and high-end.
2:54

For the chest and lapels, these layers can be attached to each other using a single-needle roll-padding machine, such as you see here. This is what you'll typically see on factory-made suits (this is a Strobel KA-ED machine). Happens both on the low- and high-end.

derek guy

257,848 görüntüleme • 11 ay önce

How many of you can drag start this Lister machine? One of my uncles who owns a grinder machine shop has the machine collected the turner and it hit his shin Broke immediately - he never walk straight again. That machine is very dangerous ❌
0:21

How many of you can drag start this Lister machine? One of my uncles who owns a grinder machine shop has the machine collected the turner and it hit his shin Broke immediately - he never walk straight again. That machine is very dangerous ❌

NaijaFarmer

290,229 görüntüleme • 11 ay önce

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.
0:56

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,741 görüntüleme • 2 yıl önce

You asked for it and now you can get @CoryWong's Clean Machine pickups today! Clean Machine pickups are overwound and carefully voiced to meet the demands of all disciples of funk and soul. Get "Clean Machine" pickups today: #SeymourDuncan #CoryWong
2:03

You asked for it and now you can get @CoryWong's Clean Machine pickups today! Clean Machine pickups are overwound and carefully voiced to meet the demands of all disciples of funk and soul. Get "Clean Machine" pickups today: #SeymourDuncan #CoryWong

Seymour Duncan

56,146 görüntüleme • 3 yıl önce

Is Hitting Off the Machine Messing You Up? Listen to Nate Headley talk about how the velo machine isn’t good for everyone and how sometimes putting it on changeup or offspeed can get your better, more natural moves to show up🗣️
0:54

Is Hitting Off the Machine Messing You Up? Listen to Nate Headley talk about how the velo machine isn’t good for everyone and how sometimes putting it on changeup or offspeed can get your better, more natural moves to show up🗣️

108 Performance

13,303 görüntüleme • 4 ay önce

$TE is trying to become the face of American solar manufacturing. Its Stage 3 layer assembly process uses 182 robots across 5 layers to build solar modules with machine-level precision designed for decades of real-world performance.
1:07

$TE is trying to become the face of American solar manufacturing. Its Stage 3 layer assembly process uses 182 robots across 5 layers to build solar modules with machine-level precision designed for decades of real-world performance.

Shay Boloor

195,997 görüntüleme • 22 gün önce

Check out the actual speed (not yet the final version) of Qwen3-Next-80B-A3B-Instruct on Apple MLX! 🔥 4-bit: 67 TPS 8-bit: 58 TPS bf16: 48 TPS Movie normal speed, only waiting times removed. Awni Hannun and Gökdeniz Gülmez did it and I bet there is still room for improvement 💪
0:27

Check out the actual speed (not yet the final version) of Qwen3-Next-80B-A3B-Instruct on Apple MLX! 🔥 4-bit: 67 TPS 8-bit: 58 TPS bf16: 48 TPS Movie normal speed, only waiting times removed. Awni Hannun and Gökdeniz Gülmez did it and I bet there is still room for improvement 💪

Ivan Fioravanti ᯅ

15,946 görüntüleme • 9 ay önce

How do you Handle the Hands of Time? Cant wait to share the rest of "Evangeline vs. The Machine" with you. 5.2.25
0:37

How do you Handle the Hands of Time? Cant wait to share the rest of "Evangeline vs. The Machine" with you. 5.2.25

Eric Church

17,754 görüntüleme • 1 yıl önce

THANK YOU @unilagnigeria FOR EVERYTHING THANK YOU MASS COMMUNICATION DEPARTMENT. EVERYTHING ABOUT THE MACHINE IS HIGHLY PRE- MEDITATED, IT JUST DOES NOT RUN ON FUEL BUT GRACE. THE MACHINE IS COMING AND IT WILL ARRIVE IN 1 PIECE 🙏🏿❤️
6:36

THANK YOU @unilagnigeria FOR EVERYTHING THANK YOU MASS COMMUNICATION DEPARTMENT. EVERYTHING ABOUT THE MACHINE IS HIGHLY PRE- MEDITATED, IT JUST DOES NOT RUN ON FUEL BUT GRACE. THE MACHINE IS COMING AND IT WILL ARRIVE IN 1 PIECE 🙏🏿❤️

INDUSTRY MACHINE ⚙️

19,910 görüntüleme • 1 yıl önce

This season, visit a vending machine that dispenses hope and help to those who are struggling, locally and around the world. With the push of a button and swipe of your card, you can donate much-needed items to give. To find a Giving Machine near you, visit:
0:45

This season, visit a vending machine that dispenses hope and help to those who are struggling, locally and around the world. With the push of a button and swipe of your card, you can donate much-needed items to give. To find a Giving Machine near you, visit:

The Church of Jesus Christ of Latter-day Saints

12,414 görüntüleme • 1 yıl önce

Thanks to President Trump, we can now see EXACTLY how the Deep State machine works.
8:54

Thanks to President Trump, we can now see EXACTLY how the Deep State machine works.

Glenn Beck

194,741 görüntüleme • 10 ay önce

I wasn't expecting such an intelligence model on my local machine. Minimax 2.1 on MLX🔥 git clone cd mlx-lm && pip install -e . mlx_lm.server --model mlx-community/MiniMax-M2.1-4bit
1:14

I wasn't expecting such an intelligence model on my local machine. Minimax 2.1 on MLX🔥 git clone cd mlx-lm && pip install -e . mlx_lm.server --model mlx-community/MiniMax-M2.1-4bit

mzba

30,377 görüntüleme • 5 ay önce

For future founders: Don’t be a cog in a machine. The machine wants you to be a cog but that’s a choice and you don’t have to give in. Be a polymath that can do many things. The world will pay 1000X more for founders who can *build* the machine.
1:49

For future founders: Don’t be a cog in a machine. The machine wants you to be a cog but that’s a choice and you don’t have to give in. Be a polymath that can do many things. The world will pay 1000X more for founders who can *build* the machine.

Garry Tan

357,729 görüntüleme • 2 yıl önce

Running GLM-4.7-Flash on 4 x M4 Pro Mac Minis using EXO Labs. Uses tensor parallelism with RDMA over Thunderbolt & MLX backend (h/t Awni Hannun). Runs at 100 tok/sec. We're working on optimizing this at EXO Labs. Aiming to hit ~200 tok/sec on this setup soon.
0:22

Running GLM-4.7-Flash on 4 x M4 Pro Mac Minis using EXO Labs. Uses tensor parallelism with RDMA over Thunderbolt & MLX backend (h/t Awni Hannun). Runs at 100 tok/sec. We're working on optimizing this at EXO Labs. Aiming to hit ~200 tok/sec on this setup soon.

Alex Cheema

62,144 görüntüleme • 5 ay önce

This is how an amazing cutting machine designed for food processors can simplify work: it's incredibly efficient and truly impressive.
1:12

This is how an amazing cutting machine designed for food processors can simplify work: it's incredibly efficient and truly impressive.

Jean Claude NIYOMUGABO

17,651,496 görüntüleme • 1 yıl önce