Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

mlx distributed uses mpi for machine to machine communication, and this is how you can handle uneven distribution of model layers. Thanks Awni Hannun for the help

digitalix's profile picture

Alex Ziskind

9,650 subscribers

31,265 Aufrufe • vor 1 Jahr •via X (Twitter)

GamingWissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von abdou
abdouvor 1 Jahr

@awnihannun v interesting, would love to know how it was ordering the machines before you specified it in the rank file. would be funny if it was just sorting the host names 😅

Profilbild von Alex Ziskind
Alex Ziskindvor 1 Jahr

@awnihannun the mysteries of mpi.

Profilbild von Rainmaker
Rainmakervor 2 Jahren

Can Machine Learning beat the market? Check out this post on my free Substack where I share code and commentary for an XGBoost model and a Random Forest model that both deliver powerful performances.

Profilbild von Mason James.3.2-12b_q4_K_S
Mason James.3.2-12b_q4_K_Svor 1 Jahr

@awnihannun Repo Prompt! ❤️ cc @pvncher

Profilbild von Art Friedman
Art Friedmanvor 1 Jahr

@awnihannun thanks for sharing! Great question for you, m3 ultra 512 vs multiple Mac mini m4 with the same budget, which option is better for LLM context?

Profilbild von Joe Burnett
Joe Burnettvor 1 Jahr

@awnihannun Awesome!

Profilbild von Mid Season Group
Mid Season Groupvor 1 Jahr

@awnihannun @Apple engineering should be ashamed.

Profilbild von Leonard Sullivan
Leonard Sullivanvor 1 Jahr

@awnihannun I love these mini racks.

Profilbild von Alex Ziskind
Alex Ziskindvor 1 Jahr

@awnihannun yep It’s a DeskPi- @Mandy_Li888

Profilbild von moskstraumen
moskstraumenvor 1 Jahr

@awnihannun Tell us more about the technical aspects, if you want!

Profilbild von CodeBreaker
CodeBreakervor 1 Jahr

@awnihannun love to see it

Ähnliche Videos

Llama 3 running locally on iPhone with MLX Built by exo intern team Mohamed Baioumy h/t Awni Hannun MLX & Prince Canuma for the port
0:24

Llama 3 running locally on iPhone with MLX Built by exo intern team Mohamed Baioumy h/t Awni Hannun MLX & Prince Canuma for the port

Alex Cheema

295,820 Aufrufe • vor 2 Jahren

Your Mac is about to run inference like a datacenter. Coming soon to MLX-Swift: Continuous batching: the fastest way to handle multiple inference streams locally. It starts with regular inference and seamlessly upgrades to batched mode when new requests arrive. The best of both worlds. Based on the work of and Awni Hannun
0:25

Your Mac is about to run inference like a datacenter. Coming soon to MLX-Swift: Continuous batching: the fastest way to handle multiple inference streams locally. It starts with regular inference and seamlessly upgrades to batched mode when new requests arrive. The best of both worlds. Based on the work of and Awni Hannun

Ronald Mannak

60,570 Aufrufe • vor 8 Monaten

oMLX is working really well as single machine inference engine for coding agents! Caching is managed perfectly (it can use a ton of disk space, be aware!) and oQ quantization delivers great results. Behind the scenes it uses the standard MLX building blocks (75% created by Prince Canuma 🙏): - mlx-lm - mlx-vlm - mlx-embeddings - mlx-audio I tested Qwen3.6-35B-A3B-oQ6 on M5 Max with two pi instances and it was fast and furious and leveraging cache like crazy as you can see in the video. Let me try to create some oQ versions (2,4,6?) of MiniMax M2.7 now and then I'll pass to distributed inference. I must win! 💪
3:57

oMLX is working really well as single machine inference engine for coding agents! Caching is managed perfectly (it can use a ton of disk space, be aware!) and oQ quantization delivers great results. Behind the scenes it uses the standard MLX building blocks (75% created by Prince Canuma 🙏): - mlx-lm - mlx-vlm - mlx-embeddings - mlx-audio I tested Qwen3.6-35B-A3B-oQ6 on M5 Max with two pi instances and it was fast and furious and leveraging cache like crazy as you can see in the video. Let me try to create some oQ versions (2,4,6?) of MiniMax M2.7 now and then I'll pass to distributed inference. I must win! 💪

Ivan Fioravanti ᯅ

19,268 Aufrufe • vor 2 Monaten

For the chest and lapels, these layers can be attached to each other using a single-needle roll-padding machine, such as you see here. This is what you'll typically see on factory-made suits (this is a Strobel KA-ED machine). Happens both on the low- and high-end.
2:54

For the chest and lapels, these layers can be attached to each other using a single-needle roll-padding machine, such as you see here. This is what you'll typically see on factory-made suits (this is a Strobel KA-ED machine). Happens both on the low- and high-end.

derek guy

257,848 Aufrufe • vor 1 Jahr

How many of you can drag start this Lister machine? One of my uncles who owns a grinder machine shop has the machine collected the turner and it hit his shin Broke immediately - he never walk straight again. That machine is very dangerous ❌
0:21

How many of you can drag start this Lister machine? One of my uncles who owns a grinder machine shop has the machine collected the turner and it hit his shin Broke immediately - he never walk straight again. That machine is very dangerous ❌

NaijaFarmer

290,229 Aufrufe • vor 1 Jahr

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.
0:56

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,745 Aufrufe • vor 2 Jahren

You asked for it and now you can get @CoryWong's Clean Machine pickups today! Clean Machine pickups are overwound and carefully voiced to meet the demands of all disciples of funk and soul. Get "Clean Machine" pickups today: #SeymourDuncan #CoryWong
2:03

You asked for it and now you can get @CoryWong's Clean Machine pickups today! Clean Machine pickups are overwound and carefully voiced to meet the demands of all disciples of funk and soul. Get "Clean Machine" pickups today: #SeymourDuncan #CoryWong

Seymour Duncan

56,173 Aufrufe • vor 3 Jahren

Is Hitting Off the Machine Messing You Up? Listen to Nate Headley talk about how the velo machine isn’t good for everyone and how sometimes putting it on changeup or offspeed can get your better, more natural moves to show up🗣️
0:54

Is Hitting Off the Machine Messing You Up? Listen to Nate Headley talk about how the velo machine isn’t good for everyone and how sometimes putting it on changeup or offspeed can get your better, more natural moves to show up🗣️

108 Performance

13,303 Aufrufe • vor 5 Monaten

$TE is trying to become the face of American solar manufacturing. Its Stage 3 layer assembly process uses 182 robots across 5 layers to build solar modules with machine-level precision designed for decades of real-world performance.
1:07

$TE is trying to become the face of American solar manufacturing. Its Stage 3 layer assembly process uses 182 robots across 5 layers to build solar modules with machine-level precision designed for decades of real-world performance.

Shay Boloor

196,618 Aufrufe • vor 1 Monat

Check out the actual speed (not yet the final version) of Qwen3-Next-80B-A3B-Instruct on Apple MLX! 🔥 4-bit: 67 TPS 8-bit: 58 TPS bf16: 48 TPS Movie normal speed, only waiting times removed. Awni Hannun and Gökdeniz Gülmez did it and I bet there is still room for improvement 💪
0:27

Check out the actual speed (not yet the final version) of Qwen3-Next-80B-A3B-Instruct on Apple MLX! 🔥 4-bit: 67 TPS 8-bit: 58 TPS bf16: 48 TPS Movie normal speed, only waiting times removed. Awni Hannun and Gökdeniz Gülmez did it and I bet there is still room for improvement 💪

Ivan Fioravanti ᯅ

15,946 Aufrufe • vor 10 Monaten

Thanks to President Trump, we can now see EXACTLY how the Deep State machine works.
8:54

Thanks to President Trump, we can now see EXACTLY how the Deep State machine works.

Glenn Beck

194,741 Aufrufe • vor 11 Monaten

For future founders: Don’t be a cog in a machine. The machine wants you to be a cog but that’s a choice and you don’t have to give in. Be a polymath that can do many things. The world will pay 1000X more for founders who can *build* the machine.
1:49

For future founders: Don’t be a cog in a machine. The machine wants you to be a cog but that’s a choice and you don’t have to give in. Be a polymath that can do many things. The world will pay 1000X more for founders who can *build* the machine.

Garry Tan

357,747 Aufrufe • vor 2 Jahren

Running GLM-4.7-Flash on 4 x M4 Pro Mac Minis using EXO Labs. Uses tensor parallelism with RDMA over Thunderbolt & MLX backend (h/t Awni Hannun). Runs at 100 tok/sec. We're working on optimizing this at EXO Labs. Aiming to hit ~200 tok/sec on this setup soon.
0:22

Running GLM-4.7-Flash on 4 x M4 Pro Mac Minis using EXO Labs. Uses tensor parallelism with RDMA over Thunderbolt & MLX backend (h/t Awni Hannun). Runs at 100 tok/sec. We're working on optimizing this at EXO Labs. Aiming to hit ~200 tok/sec on this setup soon.

Alex Cheema

62,195 Aufrufe • vor 5 Monaten

I wasn't expecting such an intelligence model on my local machine. Minimax 2.1 on MLX🔥 git clone cd mlx-lm && pip install -e . mlx_lm.server --model mlx-community/MiniMax-M2.1-4bit
1:14

I wasn't expecting such an intelligence model on my local machine. Minimax 2.1 on MLX🔥 git clone cd mlx-lm && pip install -e . mlx_lm.server --model mlx-community/MiniMax-M2.1-4bit

mzba

30,377 Aufrufe • vor 6 Monaten

This is how an amazing cutting machine designed for food processors can simplify work: it's incredibly efficient and truly impressive.
1:12

This is how an amazing cutting machine designed for food processors can simplify work: it's incredibly efficient and truly impressive.

Jean Claude NIYOMUGABO

17,651,496 Aufrufe • vor 1 Jahr