Loading video...

Video Failed to Load

Go Home

🔥Apple MLX first 6bit model is on Hugging Face!🔥 Qwen2.5-Coder-32B-Instruct-6bit! 3bit conversion and test in progress! Video 8x below on M4 Max 40GPU: - Prompt: 38 tokens, 61.731 tokens-per-sec - Generation: 1181 tokens, 16.939 tokens-per-sec - Peak memory: 25.122 GB

ivanfioravanti's profile picture

Ivan Fioravanti ᯅ

21,712 subscribers

46,494 views • 1 year ago •via X (Twitter)

Science & TechnologyNews & Politics
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

MLX + M3 Ultra 512GB + Qwen3-235B-A22B-8bit = 🔥 "write a beautiful p5js particles animation that reacts to mouse clicks movements" Prompt: 22 tokens, 64.471 tokens-per-sec Generation: 6197 tokens, 18.916 tokens-per-sec Peak memory: 251.077 GB
0:14

MLX + M3 Ultra 512GB + Qwen3-235B-A22B-8bit = 🔥 "write a beautiful p5js particles animation that reacts to mouse clicks movements" Prompt: 22 tokens, 64.471 tokens-per-sec Generation: 6197 tokens, 18.916 tokens-per-sec Peak memory: 251.077 GB

Ivan Fioravanti ᯅ

24,160 views • 1 year ago

MLX MiniMax 2.5 running LOCALLY on a single M3 Ultra 512GB! Writing a poem on LLMs at 6bit quantization! 🔥 Let's start some coding, context and distributed tests! Generation: 40.2 tokens-per-sec Peak memory: 186 GB
0:11

MLX MiniMax 2.5 running LOCALLY on a single M3 Ultra 512GB! Writing a poem on LLMs at 6bit quantization! 🔥 Let's start some coding, context and distributed tests! Generation: 40.2 tokens-per-sec Peak memory: 186 GB

Ivan Fioravanti ᯅ

226,052 views • 3 months ago

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec
0:16

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec

Ivan Fioravanti ᯅ

59,734 views • 1 year ago

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory
0:15

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory

anton

372,348 views • 2 years ago

2000 tokens/sec gives me dopamine high on opencode
0:18

2000 tokens/sec gives me dopamine high on opencode

Daniel Kim

14,504 views • 8 months ago

Latest mlx-lm has faster and lower memory prompt processing! Thanks to causal fused attention from Jagrit Digani 7B 4-bit Mistral v3 can do ~30,000 tokens under a minute on my M4 Max laptop and only needs 8.5GB:
0:17

Latest mlx-lm has faster and lower memory prompt processing! Thanks to causal fused attention from Jagrit Digani 7B 4-bit Mistral v3 can do ~30,000 tokens under a minute on my M4 Max laptop and only needs 8.5GB:

Awni Hannun

22,156 views • 1 year ago

/1 Gemma 4 31B just crushed Qwen 3.6 27B in a local LLM gamedev contest inside atomic.chat (prompt is below) Device: MacBook Pro M5 Max, 64GB RAM Results: Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens Gemma 4 31B: 27 tokens/sec · 3m 51s · 6,209 tokens So what is more important: tokens per second, or the quality of the final answer? Qwen made a very long response and showed more creativity and visual style. But Gemma gave a shorter, clearer, and more logical answer in much less time. In this one-shot Pac-Man gamedev contest, Gemma 4 31B was the clear winner. Its game logic was stronger: click reactions were smoother, and it handled interactions with elements like walls, ghosts, and particle effects better. But this was only one test. Maybe Qwen 3.6 27B can show better results with better settings. Open the comments, try our prompt, and share your result below.
0:16

/1 Gemma 4 31B just crushed Qwen 3.6 27B in a local LLM gamedev contest inside atomic.chat (prompt is below) Device: MacBook Pro M5 Max, 64GB RAM Results: Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens Gemma 4 31B: 27 tokens/sec · 3m 51s · 6,209 tokens So what is more important: tokens per second, or the quality of the final answer? Qwen made a very long response and showed more creativity and visual style. But Gemma gave a shorter, clearer, and more logical answer in much less time. In this one-shot Pac-Man gamedev contest, Gemma 4 31B was the clear winner. Its game logic was stronger: click reactions were smoother, and it handled interactions with elements like walls, ghosts, and particle effects better. But this was only one test. Maybe Qwen 3.6 27B can show better results with better settings. Open the comments, try our prompt, and share your result below.

Chubby♨️

71,368 views • 1 month ago

Introducing FlashCompact - the first specialized model for context compaction 33k tokens/sec 200k → 50k in ~1.5s Fast, high quality compaction
0:13

Introducing FlashCompact - the first specialized model for context compaction 33k tokens/sec 200k → 50k in ~1.5s Fast, high quality compaction

Morph

225,128 views • 2 months ago

The first natively trained 1-bit model: BitNet 2B. Trained on 4 trillion tokens. that can run on CPUs like Apple M2 Native 1.58-bit weights and 8bit activations W158A8 Outperforms LLaMA &close to Qwen 2.5 1.5B in while using only 0.4GB memory versus 2GB and processes tokens 40%
0:13

The first natively trained 1-bit model: BitNet 2B. Trained on 4 trillion tokens. that can run on CPUs like Apple M2 Native 1.58-bit weights and 8bit activations W158A8 Outperforms LLaMA &close to Qwen 2.5 1.5B in while using only 0.4GB memory versus 2GB and processes tokens 40%

Md Ismail Šojal 🕷️

43,617 views • 3 months ago

🦉 Owls Labs x Magic Eden 🪄 Lights are switching on 👉 Magic Eden 🪄 4,000 mintable Owls NFTs WL: Max 5 Tokens per Wallet Price: 0,08 SOL Public: Max 5 Tokens per Wallet Price: 0,12 SOL Set your alarms for 12th! ⏰
0:15

🦉 Owls Labs x Magic Eden 🪄 Lights are switching on 👉 Magic Eden 🪄 4,000 mintable Owls NFTs WL: Max 5 Tokens per Wallet Price: 0,08 SOL Public: Max 5 Tokens per Wallet Price: 0,12 SOL Set your alarms for 12th! ⏰

Owls 🦉

129,319 views • 1 year ago

GEMMA 4 FINISHED 14 MINUTES FASTER THAN QWEN 3.6 DESPITE LOWER TOKENS PER SECOND BY USING 5X FEWER TOKENS
0:16

GEMMA 4 FINISHED 14 MINUTES FASTER THAN QWEN 3.6 DESPITE LOWER TOKENS PER SECOND BY USING 5X FEWER TOKENS

0xMarioNawfal

71,957 views • 1 month ago

Qwen/Qwen3-30B-A3B is absolute game changer 🤯 ⬇️ Are you on a Mac? then use MLX weights - getting 100 tokens/sec here. It changes everything for real world use cases.
0:15

Qwen/Qwen3-30B-A3B is absolute game changer 🤯 ⬇️ Are you on a Mac? then use MLX weights - getting 100 tokens/sec here. It changes everything for real world use cases.

Victor M

142,543 views • 1 year ago

Don't miss out on HOTTEST AIRDROP! 🔥 First 10M $WHALE Tokens - Limited time available! Use Refer-a-friend program and get $WHALE Tokens, while you sleep! #crypto #airdrop #whaletoken
0:15

Sensitive content

Don't miss out on HOTTEST AIRDROP! 🔥 First 10M $WHALE Tokens - Limited time available! Use Refer-a-friend program and get $WHALE Tokens, while you sleep! #crypto #airdrop #whaletoken

Whale

10,775 views • 1 year ago

Apparently SeeDance 2 is down for the time being on BytePlus but here's a young wizard with his dragon companion. It's a very cool video model. This video cost $0.72 USD to generate which seems reasonable. (300,000 tokens for a 15 secs at $0.0024 USD per 1000 tokens.)
0:15

Apparently SeeDance 2 is down for the time being on BytePlus but here's a young wizard with his dragon companion. It's a very cool video model. This video cost $0.72 USD to generate which seems reasonable. (300,000 tokens for a 15 secs at $0.0024 USD per 1000 tokens.)

TomLikesRobots🤖

50,451 views • 3 months ago

🟪 Qwen3-235B 2507 Instruct is live on Cerebras 1400 TPS • 131K Context • 230ms TTFT $0.6 | $1.2 per M tokens Try chat: Get API key: Pay-as-you-go via
0:10

🟪 Qwen3-235B 2507 Instruct is live on Cerebras 1400 TPS • 131K Context • 230ms TTFT $0.6 | $1.2 per M tokens Try chat: Get API key: Pay-as-you-go via

Cerebras

536,400 views • 10 months ago

Pricing is simple and efficient: • 1 million tokens for $1 • The first 1 million tokens are free • No subscription or credit card needed Type “DataLumoris” in the chat for 10 million bonus tokens
0:11

Pricing is simple and efficient: • 1 million tokens for $1 • The first 1 million tokens are free • No subscription or credit card needed Type “DataLumoris” in the chat for 10 million bonus tokens

Dhaval Makwana

10,966 views • 1 year ago

First real test on M4 Max 40GPU to transcribe 3 hours of video with mlx_whisper 0.4.1 - M4 Max 2:19 mins (Max Fan no throttling) - M4 Max 2:29 mins (System Fan throttling towards end) GPU freq goes 1.578 --> 952 Here the video (8x) showing throttling kicking in at the end.
0:19

First real test on M4 Max 40GPU to transcribe 3 hours of video with mlx_whisper 0.4.1 - M4 Max 2:19 mins (Max Fan no throttling) - M4 Max 2:29 mins (System Fan throttling towards end) GPU freq goes 1.578 --> 952 Here the video (8x) showing throttling kicking in at the end.

Ivan Fioravanti ᯅ

216,328 views • 1 year ago

Imagine tokens raining down like a storm from the sky.. Use Power-Ups like: — Income per Tap — Energy Limit — Energy Recovery — Energy Consumption Get the MAX out of every tap and watch the tokens pour in. Start tapping NOW!
0:10

Imagine tokens raining down like a storm from the sky.. Use Power-Ups like: — Income per Tap — Energy Limit — Energy Recovery — Energy Consumption Get the MAX out of every tap and watch the tokens pour in. Start tapping NOW!

1win Token

57,506 views • 1 year ago

Fuck yeah! MaskGCT - New open SoTA Text to Speech model! 🔥 > Zero-shot voice cloning > Emotional TTS > Trained on 100K hours of data > Long form synthesis > Variable speed synthesis > Bilingual - Chinese & English > Available on Hugging Face Fully non-autoregressive architecture: > Stage 1: Predicts semantic tokens from text, using tokens extracted from a speech self-supervised learning (SSL) model > Stage 2: Predicts acoustic tokens conditioned on the semantic tokens. Synthesised: "Would you guys personally like to have a fake fireplace, an electric one, in your house? Or would you rather have a real fireplace? Let me know down below. Okay everybody, that's all for today's video and I hope you guys learned a bunch of furniture vocabulary!" TTS scene keeps getting lit! 🐐
0:15

Fuck yeah! MaskGCT - New open SoTA Text to Speech model! 🔥 > Zero-shot voice cloning > Emotional TTS > Trained on 100K hours of data > Long form synthesis > Variable speed synthesis > Bilingual - Chinese & English > Available on Hugging Face Fully non-autoregressive architecture: > Stage 1: Predicts semantic tokens from text, using tokens extracted from a speech self-supervised learning (SSL) model > Stage 2: Predicts acoustic tokens conditioned on the semantic tokens. Synthesised: "Would you guys personally like to have a fake fireplace, an electric one, in your house? Or would you rather have a real fireplace? Let me know down below. Okay everybody, that's all for today's video and I hope you guys learned a bunch of furniture vocabulary!" TTS scene keeps getting lit! 🐐

Vaibhav (VB) Srivastav

139,061 views • 1 year ago

We are thrilled to be a launch partner for Meta Llama 3. Experience Llama 3 now at up to 350 tokens per second for Llama 3 8B and up to 150 tokens per second for Llama 3 70B, running in full FP16 precision on the Together API! 🤯
0:10

We are thrilled to be a launch partner for Meta Llama 3. Experience Llama 3 now at up to 350 tokens per second for Llama 3 8B and up to 150 tokens per second for Llama 3 70B, running in full FP16 precision on the Together API! 🤯

Together AI

88,229 views • 2 years ago