Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

GLM-5.3-Flash streaming at 32 tok/s on a single Spark via Tensorfold ⚡️ i will share the recipe later today

filicroval's profile picture

filipe

153,537 subscribers

15,103 Aufrufe • vor 10 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Mazinho
Mazinhovor 10 Tagen

Which quant?

Profilbild von Crypterotas
Crypterotasvor 10 Tagen

Crazy What’s prefill?

Profilbild von Daniel Lougen
Daniel Lougenvor 10 Tagen

👀 you sir hav made my day

Profilbild von Kevin Luong
Kevin Luongvor 8 Tagen

Did recipe ever get posted?

Profilbild von Klasta
Klastavor 10 Tagen

glm flash already at 32 tok/s on a lone spark

Profilbild von Cato Nooka
Cato Nookavor 9 Tagen

holly molly, that's ZAI old speed at home.

Profilbild von z
zvor 10 Tagen

what quant

Profilbild von Kevin Luong
Kevin Luongvor 9 Tagen

Curious quant as well. How you fit it in 1 spark blows my mind

Profilbild von Eddi Spaghetti
Eddi Spaghettivor 10 Tagen

Wuts kld

Profilbild von Dr.-Ing. Manuel Ramsaier
Dr.-Ing. Manuel Ramsaiervor 10 Tagen

pretty nice

Ähnliche Videos

Full GLM 5.3 running on 3x DGX Sparks with TensorFold Currently 24-28 tok/s on prose
0:31

Full GLM 5.3 running on 3x DGX Sparks with TensorFold Currently 24-28 tok/s on prose

Mia

49,756 Aufrufe • vor 9 Tagen

- GLM 5.3 Flash EXL3 - 2x DGX Sparks - TensorFold - 4 concurrent prose streams Pay attention to tok/s and TTFT on each stream. Buttery smooth experience!
0:37

- GLM 5.3 Flash EXL3 - 2x DGX Sparks - TensorFold - 4 concurrent prose streams Pay attention to tok/s and TTFT on each stream. Buttery smooth experience!

Mia

32,137 Aufrufe • vor 11 Tagen

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon
0:23

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

Mia

16,605 Aufrufe • vor 1 Monat

Meet Wally. Our inference stack for open frontier models, built to be the fastest place to run them. Performance snapshot: GLM-5.3 Flash: 380 tok/s. GLM-5.3 Max: 790 tok/s Qwen3.8-27B: 485 tok/s DeepSeek-V4.1 Flash: 615 tok/s 1/7
0:46

Meet Wally. Our inference stack for open frontier models, built to be the fastest place to run them. Performance snapshot: GLM-5.3 Flash: 380 tok/s. GLM-5.3 Max: 790 tok/s Qwen3.8-27B: 485 tok/s DeepSeek-V4.1 Flash: 615 tok/s 1/7

RunAnywhere

64,347 Aufrufe • vor 20 Tagen

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.
1:10

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,628 Aufrufe • vor 1 Monat

Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s, first token in 0.46 s. Today it runs TensorFold 0.6.2: 61 tok/s, first token in 0.20 s. In the clip it laps the reply twice while the MLX side is still typing. Same reply, byte for byte. Then eight requests at once, 78 tok/s combined, every answer identical to its solo run. The old engine took one at a time. Context stays at 1,048,576 on the Mac. Every number measured on my own machine with the same probe, one week apart. Nothing estimated. Ash Hart built it. Apache 2.0.
0:32

Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s, first token in 0.46 s. Today it runs TensorFold 0.6.2: 61 tok/s, first token in 0.20 s. In the clip it laps the reply twice while the MLX side is still typing. Same reply, byte for byte. Then eight requests at once, 78 tok/s combined, every answer identical to its solo run. The old engine took one at a time. Context stays at 1,048,576 on the Mac. Every number measured on my own machine with the same probe, one week apart. Nothing estimated. Ash Hart built it. Apache 2.0.

Chad Hurley is Volatile Markets

28,044 Aufrufe • vor 9 Tagen

GLM 5.3 Locally on 8 x Blackwell 6000 @ 120 tok/s is great at making voxel dioramas One-shot prompt 👇
0:39

GLM 5.3 Locally on 8 x Blackwell 6000 @ 120 tok/s is great at making voxel dioramas One-shot prompt 👇

wick

27,982 Aufrufe • vor 1 Monat

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.
0:28

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

158,188 Aufrufe • vor 3 Monaten

We raced GLM-5.3 Flash on Wally against Z.ai 's own paid fast tier, FlashX. Same prompt, live. Wally: 363 tok/s. FlashX: 143 tok/s. 2.5x faster, 3.5x cheaper. Why are you still using slow AI? Try Wally, $5 free.
0:33

We raced GLM-5.3 Flash on Wally against Z.ai 's own paid fast tier, FlashX. Same prompt, live. Wally: 363 tok/s. FlashX: 143 tok/s. 2.5x faster, 3.5x cheaper. Why are you still using slow AI? Try Wally, $5 free.

RunAnywhere

16,410 Aufrufe • vor 16 Tagen

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.
0:20

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

Mia

24,127 Aufrufe • vor 26 Tagen

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇
0:20

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

SGLang

99,455 Aufrufe • vor 1 Monat

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in
0:22

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

atomic.chat

32,515 Aufrufe • vor 1 Monat

GLM 5.3 Flash vs Hybrid (GLM 5.3 Flash + Jev) on a turn-based strategy game (Halite 2) The hybrid approach is 13x faster, 56% of the pure GLM 5.3 Flash API cost, and gains a slight edge in performance. In the hybrid approach: -GLM 5.3 Flash handles strategic judgment of which planets matter, who expands, who fights, when to re-plan. -Jev handles the ship-by-ship execution. GLM 5.3 Flash alone defeats Jev alone 82% of the time. This is obviously just a single example and some of the first things I've tried to make use of Jev as an augmentation to an LLM. Could potentially make even more gains in cost, speed, and performance, but using what I've figured out here, I am compelled to try some ideas with robotics next.
0:24

GLM 5.3 Flash vs Hybrid (GLM 5.3 Flash + Jev) on a turn-based strategy game (Halite 2) The hybrid approach is 13x faster, 56% of the pure GLM 5.3 Flash API cost, and gains a slight edge in performance. In the hybrid approach: -GLM 5.3 Flash handles strategic judgment of which planets matter, who expands, who fights, when to re-plan. -Jev handles the ship-by-ship execution. GLM 5.3 Flash alone defeats Jev alone 82% of the time. This is obviously just a single example and some of the first things I've tried to make use of Jev as an augmentation to an LLM. Could potentially make even more gains in cost, speed, and performance, but using what I've figured out here, I am compelled to try some ideas with robotics next.

Harrison Kinsley

54,330 Aufrufe • vor 21 Tagen

Step-3.7-Flash Q4_K_S on DGX Spark (GB10, 128GB): > ~27 tok/s generation > 198B sparse MoE, ~11B active > 256K context, native vision > Agentic / tool-calling / reasoning > Apache 2.0 I added a mobile chat screen on the right showing what 27 tok/s actually feels like streaming on a phone.
0:52

Step-3.7-Flash Q4_K_S on DGX Spark (GB10, 128GB): > ~27 tok/s generation > 198B sparse MoE, ~11B active > 256K context, native vision > Agentic / tool-calling / reasoning > Apache 2.0 I added a mobile chat screen on the right showing what 27 tok/s actually feels like streaming on a phone.

stevibe

19,876 Aufrufe • vor 4 Monaten

fixed the reasoning bugs + the thinking loop in glm-5.3 flash. it's a different model now. i gave the same prompt to glm (running on my desk) vs gpt-6.1 sol. they produced a SINGLE HTML file each. very content with GLM's version, i'm genuinely impressed
0:41

fixed the reasoning bugs + the thinking loop in glm-5.3 flash. it's a different model now. i gave the same prompt to glm (running on my desk) vs gpt-6.1 sol. they produced a SINGLE HTML file each. very content with GLM's version, i'm genuinely impressed

filipe

60,225 Aufrufe • vor 7 Tagen

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle
0:43

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

Abdur Rahim

110,412 Aufrufe • vor 27 Tagen

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.
0:37

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

Awni Hannun

141,194 Aufrufe • vor 8 Monaten

🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. Alex Ziskind Thanks for letting me use the M5Us!
0:26

🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. Alex Ziskind Thanks for letting me use the M5Us!

Ash Hart

26,047 Aufrufe • vor 5 Tagen

GLM-5.3 Flash in already nerfed? I compared Ox Alpha in it's prime vs today just before it got revealed as GLM-5.3 Flash using the same prompt to build 3d raptor engine using three.js OxAlpha on 24th August gave far more accurate output compared to today which looks way worse prompt i used: Create a 3d SpaceX raptor engine using three.js, use white theme background so the main object is clearly visible
0:28

GLM-5.3 Flash in already nerfed? I compared Ox Alpha in it's prime vs today just before it got revealed as GLM-5.3 Flash using the same prompt to build 3d raptor engine using three.js OxAlpha on 24th August gave far more accurate output compared to today which looks way worse prompt i used: Create a 3d SpaceX raptor engine using three.js, use white theme background so the main object is clearly visible

Tim Jayas

82,975 Aufrufe • vor 1 Monat

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)
1:02

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Killy

30,081 Aufrufe • vor 1 Monat