Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

GLM-5.3-Flash streaming at 32 tok/s on a single Spark via Tensorfold ⚡️ i will share the recipe later today

filicroval's profile picture

filipe

153,537 subscribers

15,103 görüntüleme • 10 gün önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

10 Yorum

Mazinho profil fotoğrafı
Mazinho10 gün önce

Which quant?

Crypterotas profil fotoğrafı
Crypterotas10 gün önce

Crazy What’s prefill?

Daniel Lougen profil fotoğrafı
Daniel Lougen10 gün önce

👀 you sir hav made my day

Kevin Luong profil fotoğrafı
Kevin Luong8 gün önce

Did recipe ever get posted?

Klasta profil fotoğrafı
Klasta10 gün önce

glm flash already at 32 tok/s on a lone spark

Cato Nooka profil fotoğrafı
Cato Nooka9 gün önce

holly molly, that's ZAI old speed at home.

z profil fotoğrafı
z10 gün önce

what quant

Kevin Luong profil fotoğrafı
Kevin Luong9 gün önce

Curious quant as well. How you fit it in 1 spark blows my mind

Eddi Spaghetti profil fotoğrafı
Eddi Spaghetti10 gün önce

Wuts kld

Dr.-Ing. Manuel Ramsaier profil fotoğrafı
Dr.-Ing. Manuel Ramsaier10 gün önce

pretty nice

Benzer Videolar

Full GLM 5.3 running on 3x DGX Sparks with TensorFold Currently 24-28 tok/s on prose
0:31

Full GLM 5.3 running on 3x DGX Sparks with TensorFold Currently 24-28 tok/s on prose

Mia

49,756 görüntüleme • 9 gün önce

- GLM 5.3 Flash EXL3 - 2x DGX Sparks - TensorFold - 4 concurrent prose streams Pay attention to tok/s and TTFT on each stream. Buttery smooth experience!
0:37

- GLM 5.3 Flash EXL3 - 2x DGX Sparks - TensorFold - 4 concurrent prose streams Pay attention to tok/s and TTFT on each stream. Buttery smooth experience!

Mia

32,137 görüntüleme • 11 gün önce

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon
0:23

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

Mia

16,605 görüntüleme • 1 ay önce

Meet Wally. Our inference stack for open frontier models, built to be the fastest place to run them. Performance snapshot: GLM-5.3 Flash: 380 tok/s. GLM-5.3 Max: 790 tok/s Qwen3.8-27B: 485 tok/s DeepSeek-V4.1 Flash: 615 tok/s 1/7
0:46

Meet Wally. Our inference stack for open frontier models, built to be the fastest place to run them. Performance snapshot: GLM-5.3 Flash: 380 tok/s. GLM-5.3 Max: 790 tok/s Qwen3.8-27B: 485 tok/s DeepSeek-V4.1 Flash: 615 tok/s 1/7

RunAnywhere

64,347 görüntüleme • 20 gün önce

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.
1:10

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,628 görüntüleme • 1 ay önce

Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s, first token in 0.46 s. Today it runs TensorFold 0.6.2: 61 tok/s, first token in 0.20 s. In the clip it laps the reply twice while the MLX side is still typing. Same reply, byte for byte. Then eight requests at once, 78 tok/s combined, every answer identical to its solo run. The old engine took one at a time. Context stays at 1,048,576 on the Mac. Every number measured on my own machine with the same probe, one week apart. Nothing estimated. Ash Hart built it. Apache 2.0.
0:32

Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s, first token in 0.46 s. Today it runs TensorFold 0.6.2: 61 tok/s, first token in 0.20 s. In the clip it laps the reply twice while the MLX side is still typing. Same reply, byte for byte. Then eight requests at once, 78 tok/s combined, every answer identical to its solo run. The old engine took one at a time. Context stays at 1,048,576 on the Mac. Every number measured on my own machine with the same probe, one week apart. Nothing estimated. Ash Hart built it. Apache 2.0.

Chad Hurley is Volatile Markets

28,044 görüntüleme • 9 gün önce

GLM 5.3 Locally on 8 x Blackwell 6000 @ 120 tok/s is great at making voxel dioramas One-shot prompt 👇
0:39

GLM 5.3 Locally on 8 x Blackwell 6000 @ 120 tok/s is great at making voxel dioramas One-shot prompt 👇

wick

27,982 görüntüleme • 1 ay önce

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.
0:28

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

158,188 görüntüleme • 3 ay önce

We raced GLM-5.3 Flash on Wally against Z.ai 's own paid fast tier, FlashX. Same prompt, live. Wally: 363 tok/s. FlashX: 143 tok/s. 2.5x faster, 3.5x cheaper. Why are you still using slow AI? Try Wally, $5 free.
0:33

We raced GLM-5.3 Flash on Wally against Z.ai 's own paid fast tier, FlashX. Same prompt, live. Wally: 363 tok/s. FlashX: 143 tok/s. 2.5x faster, 3.5x cheaper. Why are you still using slow AI? Try Wally, $5 free.

RunAnywhere

16,410 görüntüleme • 16 gün önce

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.
0:20

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

Mia

24,127 görüntüleme • 26 gün önce

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇
0:20

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

SGLang

99,455 görüntüleme • 1 ay önce

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in
0:22

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

atomic.chat

32,515 görüntüleme • 1 ay önce

GLM 5.3 Flash vs Hybrid (GLM 5.3 Flash + Jev) on a turn-based strategy game (Halite 2) The hybrid approach is 13x faster, 56% of the pure GLM 5.3 Flash API cost, and gains a slight edge in performance. In the hybrid approach: -GLM 5.3 Flash handles strategic judgment of which planets matter, who expands, who fights, when to re-plan. -Jev handles the ship-by-ship execution. GLM 5.3 Flash alone defeats Jev alone 82% of the time. This is obviously just a single example and some of the first things I've tried to make use of Jev as an augmentation to an LLM. Could potentially make even more gains in cost, speed, and performance, but using what I've figured out here, I am compelled to try some ideas with robotics next.
0:24

GLM 5.3 Flash vs Hybrid (GLM 5.3 Flash + Jev) on a turn-based strategy game (Halite 2) The hybrid approach is 13x faster, 56% of the pure GLM 5.3 Flash API cost, and gains a slight edge in performance. In the hybrid approach: -GLM 5.3 Flash handles strategic judgment of which planets matter, who expands, who fights, when to re-plan. -Jev handles the ship-by-ship execution. GLM 5.3 Flash alone defeats Jev alone 82% of the time. This is obviously just a single example and some of the first things I've tried to make use of Jev as an augmentation to an LLM. Could potentially make even more gains in cost, speed, and performance, but using what I've figured out here, I am compelled to try some ideas with robotics next.

Harrison Kinsley

54,330 görüntüleme • 21 gün önce

Step-3.7-Flash Q4_K_S on DGX Spark (GB10, 128GB): > ~27 tok/s generation > 198B sparse MoE, ~11B active > 256K context, native vision > Agentic / tool-calling / reasoning > Apache 2.0 I added a mobile chat screen on the right showing what 27 tok/s actually feels like streaming on a phone.
0:52

Step-3.7-Flash Q4_K_S on DGX Spark (GB10, 128GB): > ~27 tok/s generation > 198B sparse MoE, ~11B active > 256K context, native vision > Agentic / tool-calling / reasoning > Apache 2.0 I added a mobile chat screen on the right showing what 27 tok/s actually feels like streaming on a phone.

stevibe

19,876 görüntüleme • 4 ay önce

fixed the reasoning bugs + the thinking loop in glm-5.3 flash. it's a different model now. i gave the same prompt to glm (running on my desk) vs gpt-6.1 sol. they produced a SINGLE HTML file each. very content with GLM's version, i'm genuinely impressed
0:41

fixed the reasoning bugs + the thinking loop in glm-5.3 flash. it's a different model now. i gave the same prompt to glm (running on my desk) vs gpt-6.1 sol. they produced a SINGLE HTML file each. very content with GLM's version, i'm genuinely impressed

filipe

60,225 görüntüleme • 7 gün önce

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle
0:43

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

Abdur Rahim

110,412 görüntüleme • 27 gün önce

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.
0:37

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

Awni Hannun

141,194 görüntüleme • 8 ay önce

🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. Alex Ziskind Thanks for letting me use the M5Us!
0:26

🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. Alex Ziskind Thanks for letting me use the M5Us!

Ash Hart

26,047 görüntüleme • 5 gün önce

GLM-5.3 Flash in already nerfed? I compared Ox Alpha in it's prime vs today just before it got revealed as GLM-5.3 Flash using the same prompt to build 3d raptor engine using three.js OxAlpha on 24th August gave far more accurate output compared to today which looks way worse prompt i used: Create a 3d SpaceX raptor engine using three.js, use white theme background so the main object is clearly visible
0:28

GLM-5.3 Flash in already nerfed? I compared Ox Alpha in it's prime vs today just before it got revealed as GLM-5.3 Flash using the same prompt to build 3d raptor engine using three.js OxAlpha on 24th August gave far more accurate output compared to today which looks way worse prompt i used: Create a 3d SpaceX raptor engine using three.js, use white theme background so the main object is clearly visible

Tim Jayas

82,975 görüntüleme • 1 ay önce

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)
1:02

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Killy

30,081 görüntüleme • 1 ay önce