Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

- GLM 5.3 Flash EXL3 - 2x DGX Sparks - TensorFold - 4 concurrent prose streams Pay attention to tok/s and TTFT on each stream. Buttery smooth experience!

Mia

40,234 subscribers

32,137 views • 11 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

39 Comments

Ash Hart11 days ago

Excellent work on this recipe, Mia! 🚀

Mark Argyelan11 days ago

Already installed! Thanks, @ashxhart & @MiaAI_lab The numbers speak for themselves: both Sparks CC@2000, with very low power consumption. to be exact I measured 37-45W Best Package so far!!

Matt Barrie11 days ago

You can see this is going to be a massive problem for Anthropic & Open AI

just mntz11 days ago

Great! What about intelligence? Can it keep up with NVFP4?

Mia11 days ago

no. it's better.

demmon standafer11 days ago

@just_mntz Why? Isn’t nvpf4 more optimized?

Mia11 days ago

@just_mntz noup

Mackross11 days ago

This is unbelievable. Congratulations to you and Ash

JCode11 days ago

looks amazing! also the harness is looking great Mia!

Mia11 days ago

thanks!

Daniel Banu11 days ago

Is there anything lost by going to fp8 kv cache?

Mia11 days ago

None, I've tested it

Sayyid Fareed11 days ago

Beautiful recipe. Fast and completely fixes the issue we had on the previous Tensorfold recipe last night where the worker would overheat due to idling on the GPU. Amazing work!

Norman Headings11 days ago

Awesome stuff! Any plans to use tensorfold on ds 4.1 ?

Mia11 days ago

ofc

🦁 Lion (LFMN)11 days ago

Amazing 🔥🔥🔥

Daniel Morris11 days ago

I will be joining the 2x DGX spark club shortly

pd1911 days ago

还是你的工具好看，老妹儿

☁️11 days ago

Ablit?

Mia11 days ago

Soon

Soveryn Intelligence11 days ago

Ours, 4 streams: 42.6 tok/s each, TTFT 312–313 ms on every slot. Old vLLM Kernel was 10.9 each at ~2.6 s. Stream 4 doesn't bleed.

Mia11 days ago

You get BETTER speeds than me on MY recipe?? lol

Soveryn Intelligence11 days ago

hey what can I tell ya maybe its the harness... we are running it on Pi our own version of soveryn pi.. but with out your set up I would not have any numbers to share really nice work

hhh ppp11 days ago

I really need proper images handling, 4 is not enough and aplying the image limits patch would reprocess the entire conversation, any plans for a fix for this?

Sidhanth Krishna11 days ago

Legend 🫡 Tried it out and works as expected. Noticed the following and was hoping to get your two cents on the issue: - Prompt cache is breaking with image inputs on opencode (haven’t tried with videos) - Rig is running hot during prefill, should we decrease the clock?

Liam Nerd11 days ago

Four streams is the workday case.

gQ11 days ago

Stability and speed are looking amazing. Congrats to everyone involved 👏

Cesar A.11 days ago

If you were buying today would you go 2x DGX Sparks or M5 Ultra 256GB?

Alissa11 days ago

Prefill is at ~840 tok/s, not the ~1,950 in the README. Their own benchmark tool measures the same here. any hints ?

sic Fona11 days ago

もはやクラウド級のAI🤯

James Malsawm11 days ago

Amazing

Dirk Glensk11 days ago

Nice! Can you also show one where you input large text + images?

basedcapital11 days ago

2x dgx spark is 256gb of unified memory on a desk. tensorfold's whole trick is lazy mmap streaming so you never pin all the weights at once, and exl3 shrinks the quants further. 4 concurrent streams feeling buttery makes sense, the box finally has headroom for kv cache.

Me&Doggo11 days ago

The speed, and the speed of new changes, is amazing.

Yacine Zaitri11 days ago

How about running this on a single ultra5? How is TensorFold compared with MLX?

L1inear11 days ago

Any chance you'll optimize MLX once 512 GB ultras get released?

Alissa11 days ago

great work ! trying it now and comparing with the vllm 2xsparks vs 2xsparks

Aditya11 days ago

This is great look at the improvement! Thanks @MiaAI_lab

A11 days ago

I don’t think people outside of local AI comprehend what huge advancement we’ve seen in the past few months. It’s absolutely incredible. Thank you so much for all the work you’ve done.

Related Videos

How prose looks like in Hermes agent using GLM 5.3 Flash EXL3 on 2x DGX Sparks

How prose looks like in Hermes agent using GLM 5.3 Flash EXL3 on 2x DGX Sparks

Mia

10,574 views • 1 month ago

Full GLM 5.3 running on 3x DGX Sparks with TensorFold Currently 24-28 tok/s on prose

Full GLM 5.3 running on 3x DGX Sparks with TensorFold Currently 24-28 tok/s on prose

Mia

49,756 views • 9 days ago

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

Mia

16,605 views • 1 month ago

GLM 5.3 Flash EXL3 running on 3x DGX Sparks.

GLM 5.3 Flash EXL3 running on 3x DGX Sparks.

Mia

12,535 views • 27 days ago

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,628 views • 1 month ago

How DeepSeek v4.1 Flash looks on 3x DGX Sparks with 4 concurrent streams

How DeepSeek v4.1 Flash looks on 3x DGX Sparks with 4 concurrent streams

Mia

19,730 views • 1 month ago

GLM-5.3-Flash streaming at 32 tok/s on a single Spark via Tensorfold ⚡️ i will share the recipe later today

GLM-5.3-Flash streaming at 32 tok/s on a single Spark via Tensorfold ⚡️ i will share the recipe later today

filipe

15,103 views • 10 days ago

Check out how DeepSeek v4.1 Flash performs on 2x DGX Sparks after the update. Smooth experience!

Check out how DeepSeek v4.1 Flash performs on 2x DGX Sparks after the update. Smooth experience!

Mia

39,491 views • 26 days ago

This is what 20 tok/s looks like on 4 x DGX Sparks running GLM 5.2

This is what 20 tok/s looks like on 4 x DGX Sparks running GLM 5.2

Tech2Wild

14,140 views • 3 months ago

I ran this locally with GLM-5.3 Flash EXL3 on my DGX Sparks, and I think it beats both Fable 5.1 and GPT-6 Astra. The crazy part: it was faster to make create this on my sparks than it was with either Fable 5.1 or Astra. Live link here:

I ran this locally with GLM-5.3 Flash EXL3 on my DGX Sparks, and I think it beats both Fable 5.1 and GPT-6 Astra. The crazy part: it was faster to make create this on my sparks than it was with either Fable 5.1 or Astra. Live link here:

Mia

48,031 views • 1 month ago

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Killy

30,081 views • 1 month ago

🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. Alex Ziskind Thanks for letting me use the M5Us!

🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. Alex Ziskind Thanks for letting me use the M5Us!

Ash Hart

26,047 views • 5 days ago

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!

Ronald Mannak

53,392 views • 5 months ago

Loading DeepSeek v4 Flash just reminded me how good and fast this model is and how well it runs on DGX Sparks. GLM-5.2 is really awesome, but if you want fast and concurrent inference, DS4F is still a top choice IMO. Created by DS4F in single shot The coffee guy

Loading DeepSeek v4 Flash just reminded me how good and fast this model is and how well it runs on DGX Sparks. GLM-5.2 is really awesome, but if you want fast and concurrent inference, DS4F is still a top choice IMO. Created by DS4F in single shot The coffee guy

Mia

40,899 views • 2 months ago

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

stevibe

403,356 views • 5 months ago

Qwen3.6 27B landed yesterday, so I ran it on 4 setups side-by-side to see how they stack up: 🔴 RTX 4090 — 45.59 tok/s, TTFT 525ms 🟢 RTX 5090 — 51.83 tok/s, TTFT 752ms ⚫️ M2 Ultra — 22.30 tok/s, TTFT 216ms 🟣 DGX Spark — 11.08 tok/s, TTFT 319ms This is a standard test: no tuning, just the out-of-the-box experience. For the NVIDIA cards I used llama.cpp with Unsloth's UD-Q4_K_XL quant. For the M2 Ultra I used MLX with Unsloth's UD-MLX-4bit quant, since MLX is the native path on Apple Silicon. Please consider this as the baseline, you can definitely squeeze more out of every one of these with fine-tuned settings.

Qwen3.6 27B landed yesterday, so I ran it on 4 setups side-by-side to see how they stack up: 🔴 RTX 4090 — 45.59 tok/s, TTFT 525ms 🟢 RTX 5090 — 51.83 tok/s, TTFT 752ms ⚫️ M2 Ultra — 22.30 tok/s, TTFT 216ms 🟣 DGX Spark — 11.08 tok/s, TTFT 319ms This is a standard test: no tuning, just the out-of-the-box experience. For the NVIDIA cards I used llama.cpp with Unsloth's UD-Q4_K_XL quant. For the M2 Ultra I used MLX with Unsloth's UD-MLX-4bit quant, since MLX is the native path on Apple Silicon. Please consider this as the baseline, you can definitely squeeze more out of every one of these with fine-tuned settings.

stevibe

104,692 views • 5 months ago

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

SGLang

99,455 views • 1 month ago

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

atomic.chat

32,515 views • 1 month ago

🚨 We finally cracked it. GLM-5.3, the FULL model (not the Flash), running on 4 DGX Sparks 🔥 ⚡️ 35 tok/s on real coding, peaking near 40 when it's hot 📊 Full C1 to C6 benchmark done, same speeds we were hitting on 5.2 🧠 200K context right now, tuning it higher from here 🎯 And this is only MTP4... we haven't even added DFlash yet I quantized it myself and I'm uploading the repo RIGHT NOW so everyone can go tune it with me 👀 Be tuned, it drops any minute. Let's get it 🚀

🚨 We finally cracked it. GLM-5.3, the FULL model (not the Flash), running on 4 DGX Sparks 🔥 ⚡️ 35 tok/s on real coding, peaking near 40 when it's hot 📊 Full C1 to C6 benchmark done, same speeds we were hitting on 5.2 🧠 200K context right now, tuning it higher from here 🎯 And this is only MTP4... we haven't even added DFlash yet I quantized it myself and I'm uploading the repo RIGHT NOW so everyone can go tune it with me 👀 Be tuned, it drops any minute. Let's get it 🚀

Tech2Wild

13,377 views • 1 month ago

Everyone's comparing the DGX Spark to a 5090 and calling it slow. I think that's the wrong comparison. I ran Qwen3.6 35B-A3B FP8 with the full 262K context window enabled (~96GB RAM) — something gaming GPUs can't really do. Results: 🟢No context: 51.3 tok/s, TTFT 110ms 🟣200K prefill: 34.6 tok/s, TTFT 85s (~2,341 tok/s prefill) Prefill is way faster than a Mac. And 35 tok/s deep into 200K context, on a model this strong, is genuinely usable. The Spark plays a different game.

Everyone's comparing the DGX Spark to a 5090 and calling it slow. I think that's the wrong comparison. I ran Qwen3.6 35B-A3B FP8 with the full 262K context window enabled (~96GB RAM) — something gaming GPUs can't really do. Results: 🟢No context: 51.3 tok/s, TTFT 110ms 🟣200K prefill: 34.6 tok/s, TTFT 85s (~2,341 tok/s prefill) Prefill is way faster than a Mac. And 35 tok/s deep into 200K context, on a model this strong, is genuinely usable. The Spark plays a different game.

stevibe

34,141 views • 5 months ago