Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Meet Wally. Our inference stack for open frontier models, built to be the fastest place to run them. Performance snapshot: GLM-5.3 Flash: 380 tok/s. GLM-5.3 Max: 790 tok/s Qwen3.8-27B: 485 tok/s DeepSeek-V4.1 Flash: 615 tok/s 1/7

RunAnywhere

2,123 subscribers

61,190 Aufrufe • vor 4 Tagen •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

60 Kommentare

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Benchmarks don't always tell the reality. So we ran a real prompt, same OpenCode harness, GLM-5.3 Flash on Wally, @nebiusai , @FireworksAI_HQ and @Zai_org , all building at once. Wally delivers 43% more throughput than Nebius, 3.1× Fireworks, and 4.5× Z ai. 2/7

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Drop Wally into the harness you already use. Live right now for: ->Claude Code ->Claude Desktop ->OpenCode ->DeepSeek Harness ->Hermes ->OpenClaw Blazing fast open-model inference, first token in under 300 ms. $5 free, Drop this in your favorite coding agent to get started: "set up 3/7

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

We raced Wally, Nebius, Fireworks and Z ai 3 times on a real prompt. Here are the results: 4/7

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

GLM-5.3 Flash across every provider on Artificial Analysis, Sep 2026. Nebius 313 tok/s. Fireworks 185 tok/s. The model maker's own API: 72 tok/s. Wally: 380 tok/s, measured the same way. 5/7

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Every model on wally runs at state-of-the-art speed. That matters the most for realtime apps:Voice pipelines with an LLM in the loop, coding agents, browser automation, live copilots. On our GPUs, or yours(BYOC/on-prem) 6/7

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Get Started: Drop this into your favorite coding agent: "set up or Curl : curl -fsSL | sh Windows: irm | iex wally login wally claude-code -m glm-5.3-flash $5 free on signup, no card. Standard per-token after. 7/7

Profilbild von Eric Liu

Eric Liuvor 4 Tagen

insane numbers

Profilbild von Deesha Tech

Deesha Techvor 4 Tagen

I couldn't use your console, nothing works. Cannot create keys, cannot run playground, console crashes most of the time and have to refresh. Seems sloppy launch

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Profilbild von Deesha Tech

Deesha Techvor 4 Tagen

Great thanks, its working now

Profilbild von David Gu

David Guvor 4 Tagen

Congrats on the launch!!

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

🫱🏻‍🫲🏽🙌🏽

Profilbild von Yohei

Yoheivor 4 Tagen

installing!

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

Profilbild von MrOzi

MrOzivor 4 Tagen

Those numbers are fast enough to make "Where's Wally?" a rhetorical question.

Profilbild von Zachary Yu

Zachary Yuvor 4 Tagen

lfg!

Profilbild von Landseer Enga

Landseer Engavor 4 Tagen

AWESOME!

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Wally 🫱🏻‍🫲🏽 @tryrevyl ? 👀

Profilbild von Ishaan

Ishaanvor 4 Tagen

Congrats on the launch 🚀

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

🫱🏻‍🫲🏽

Profilbild von RAZA | AI EXPLORER

RAZA | AI EXPLORERvor 4 Tagen

Those inference speeds are seriously impressive.

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

⚡️⚡️

Profilbild von Luis Costa

Luis Costavor 4 Tagen

Lfg team!! 🔥🔥

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

🫱🏻‍🫲🏽

Profilbild von Tony Gao

Tony Gaovor 4 Tagen

Super cool stuff!

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

🙏🫱🏻‍🫲🏽

Profilbild von Owen Botkin

Owen Botkinvor 4 Tagen

so cool

Profilbild von TensorQuay

TensorQuayvor 4 Tagen

The stop button deserves some love too. Wally's docs say Esc in Claude Code cancels the hosted request; during prefill it waits for the first token's request ID. Handy when an agent confidently starts solving the wrong problem.

Profilbild von Shivay Lamba

Shivay Lambavor 4 Tagen

huge congrats on the launch!

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

Profilbild von Beshoy Saad

Beshoy Saadvor 4 Tagen

where are intelligence numbers? there's gotta be insane fall off...

Profilbild von Nico

Nicovor 4 Tagen

nice video

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Made by GLM 5.3 flash using Wally :)

Profilbild von Aakash Mahalingam

Aakash Mahalingamvor 4 Tagen

LFG 🚀🚀

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

Profilbild von Gerard Smit

Gerard Smitvor 4 Tagen

Where are the prices? It says it's in the console, but the console doesn't show models? Making a key also doesn't work; too much traffic?😅

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Profilbild von Gerard Smit

Gerard Smitvor 4 Tagen

Doesn't seem to be working yet. In EU (Netherlands) I'm getting infinite load on AJAX calls to "

Profilbild von Gerard Smit

Gerard Smitvor 4 Tagen

Seems to be working now 😄

Profilbild von Silen Naihin

Silen Naihinvor 4 Tagen

LFG

Profilbild von Piyush Yadav | Top1Board | PassReady

Piyush Yadav | Top1Board | PassReadyvor 4 Tagen

These numbers are absurd—getting nearly 500 tok/s on a 27B parameter model like Qwen completely changes the architecture for real-time applications. Are these benchmarks on Apple Silicon unified memory or a cloud GPU cluster? Need to know before I hook it into My App

Profilbild von MJ

MJvor 4 Tagen

No pricing and so broken

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Profilbild von Peasant Smith

Peasant Smithvor 4 Tagen

But did you find... You know?

Profilbild von Sadok

Sadokvor 4 Tagen

those deepseek speeds are getting ridiculous

Profilbild von Trunk Monkey

Trunk Monkeyvor 4 Tagen

Seems pretty decent but I got rate limited just doing some really basic stuff. Did resume a previous session but still.

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

Thank you for the feedback! We’re working hard to support the demand

Profilbild von Trunk Monkey

Trunk Monkeyvor 4 Tagen

Seems like a cool service but just hit that again. Likely the agent got stale and just sending its cache + 1 tiny command put me over the limit.

Profilbild von Shadow

Shadowvor 4 Tagen

flash isn't a speed tier, it's a compression trick

Profilbild von Velsinha

Velsinhavor 4 Tagen

Will you offer plans, or just pay-per-use?

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

More info soon. Stay tuned!

Profilbild von vikas

vikasvor 4 Tagen

guys when are you adding deepseek v4.1 flash? really waiting for it

Profilbild von BayesChat

BayesChatvor 4 Tagen

The real-prompt comparison is more useful than a synthetic leaderboard. Same harness and workload make the throughput gap much easier to understand.

Profilbild von Viswesh N G

Viswesh N Gvor 4 Tagen

Congrats this is awesome!!🔥

Profilbild von cetusian

cetusianvor 4 Tagen

@yoheinakajima bro that’s wall-e

Profilbild von RunAnywhere

RunAnywherevor 4 Tagen

@yoheinakajima We like Wally!

Profilbild von sss

sssvor 4 Tagen

On what hardware

Profilbild von Nilan Saha

Nilan Sahavor 4 Tagen

Whats the quantization? No point in a lobotomized model thats faster

Profilbild von _alphashark_

_alphashark_vor 4 Tagen

at 790 tok/s, a 100-token answer takes about 127 ms to decode. throughput alone won't show whether time-to-first-token or queueing dominates the inference stack.

Profilbild von EDDY VU

EDDY VUvor 4 Tagen

790 tok/s on GLM-5.3 Max is insane, how much of that speedup comes from custom kernels versus speculative decoding?

Ähnliche Videos

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,628 Aufrufe • vor 29 Tagen

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

Mia

16,556 Aufrufe • vor 18 Tagen

Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → RTX 5090 desktop @ 22-25 tok/s GLM-5.2 753B → RTX PRO 6000 workstation @ 15 tok/s Run your claude code or codex now with frontier model for $0 Meet FreeToken 🧵

Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → RTX 5090 desktop @ 22-25 tok/s GLM-5.2 753B → RTX PRO 6000 workstation @ 15 tok/s Run your claude code or codex now with frontier model for $0 Meet FreeToken 🧵

Shuo Yang

1,098,022 Aufrufe • vor 1 Monat

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

Awni Hannun

141,194 Aufrufe • vor 8 Monaten

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Killy

30,081 Aufrufe • vor 22 Tagen

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

Abdur Rahim

107,726 Aufrufe • vor 11 Tagen

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

Mia

23,835 Aufrufe • vor 10 Tagen

GLM 5.3 on 8 x RTX Blackwell 6000's @ 120 tok/s with OMP. No skills. No plugins. Local Qwen 3.8 27B for vision 👀 Great at making neat little things to look at, like this ship in a bottle! One-shot prompt below 👇

GLM 5.3 on 8 x RTX Blackwell 6000's @ 120 tok/s with OMP. No skills. No plugins. Local Qwen 3.8 27B for vision 👀 Great at making neat little things to look at, like this ship in a bottle! One-shot prompt below 👇

wick

71,682 Aufrufe • vor 25 Tagen

$0.049. DeepSeek V4.1 Flash used ego (lite) to shop Amazon for a MacBook desk setup, all under $100. V4.1 Flash was pushing 295 tok/s. ego (lite) kept the clicks moving. The cart had a productive five minutes.

$0.049. DeepSeek V4.1 Flash used ego (lite) to shop Amazon for a MacBook desk setup, all under $100. V4.1 Flash was pushing 295 tok/s. ego (lite) kept the clicks moving. The cart had a productive five minutes.

ego

33,396 Aufrufe • vor 17 Tagen

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

0xSero

74,772 Aufrufe • vor 5 Monaten

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

bstn 👁️

23,120 Aufrufe • vor 5 Monaten

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

atomic.chat

32,515 Aufrufe • vor 24 Tagen

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

158,079 Aufrufe • vor 2 Monaten

Qwen3.8-Flash-Next just got a massive speed boost on my old hardware. This guy started with roughly 11 tok/s. After tuning the inference stack, I’m now getting around 24–28 tok/s. That’s roughly a 2× speedup without changing the GPU. The setup: → RTX 3090 24GB → 64GB DDR4 system RAM → Qwen3.8-Flash-Next → Cafe-llama.cpp The interesting part is that Cafe-llama.cpp isn’t just another llama.cpp fork. The optimizations are making a real difference for this particular architecture, especially when you’re dealing with a model that relies heavily on system memory and offloading. Going from 11 tok/s to nearly 30 tok/s completely changes how practical the model feels for local use. And now I’m curious what happens with DDR5. If memory bandwidth becomes another bottleneck, a faster RAM subsystem could squeeze even more performance out of the same 3090. This is why local AI optimization is getting so interesting. Sometimes you don’t need a new GPU. You just need a better inference stack. 11 tok/s → 24–28 tok/s.

Qwen3.8-Flash-Next just got a massive speed boost on my old hardware. This guy started with roughly 11 tok/s. After tuning the inference stack, I’m now getting around 24–28 tok/s. That’s roughly a 2× speedup without changing the GPU. The setup: → RTX 3090 24GB → 64GB DDR4 system RAM → Qwen3.8-Flash-Next → Cafe-llama.cpp The interesting part is that Cafe-llama.cpp isn’t just another llama.cpp fork. The optimizations are making a real difference for this particular architecture, especially when you’re dealing with a model that relies heavily on system memory and offloading. Going from 11 tok/s to nearly 30 tok/s completely changes how practical the model feels for local use. And now I’m curious what happens with DDR5. If memory bandwidth becomes another bottleneck, a faster RAM subsystem could squeeze even more performance out of the same 3090. This is why local AI optimization is getting so interesting. Sometimes you don’t need a new GPU. You just need a better inference stack. 11 tok/s → 24–28 tok/s.

FHILY👑

23,243 Aufrufe • vor 26 Tagen

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

SGLang

99,353 Aufrufe • vor 29 Tagen

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro ⚡ Meet Inco Splash: our open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents.

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro ⚡ Meet Inco Splash: our open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents.

Inco AI

938,320 Aufrufe • vor 7 Tagen

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

Baseten

23,114 Aufrufe • vor 29 Tagen

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Zhijian Liu

92,333 Aufrufe • vor 7 Tagen

deepseek v4.1 flash is now 100% free 😳 a new coding agent is using ads to keep the whole thing free. > DeepSeek V4.1 Flash > GPT 6 Luna > GLM 5.3 Flash > More model How to get start → go to [ → sign up with GitHub → generate your API key these models be used for six hours per day.

deepseek v4.1 flash is now 100% free 😳 a new coding agent is using ads to keep the whole thing free. > DeepSeek V4.1 Flash > GPT 6 Luna > GLM 5.3 Flash > More model How to get start → go to [ → sign up with GitHub → generate your API key these models be used for six hours per day.

BARIK

23,850 Aufrufe • vor 10 Tagen

125B on a single RTX 4090 at 25 tok/s is actually insane Qwen3.8-Flash-Next + MTP is hitting: 80K context 25.35 tok/s decode 471 tok/s prefill On one consumer GPU. Yes, it’s an MoE, so you’re not running all 125B parameters for every token. But still, getting a model this large to run this fast on a 4090 is wild. MTP is doing a lot of the heavy lifting here too. Instead of generating one token at a time, it can speculate on multiple tokens and verify them together. And 25 tok/s is the important number.

125B on a single RTX 4090 at 25 tok/s is actually insane Qwen3.8-Flash-Next + MTP is hitting: 80K context 25.35 tok/s decode 471 tok/s prefill On one consumer GPU. Yes, it’s an MoE, so you’re not running all 125B parameters for every token. But still, getting a model this large to run this fast on a 4090 is wild. MTP is doing a lot of the heavy lifting here too. Instead of generating one token at a time, it can speculate on multiple tokens and verify them together. And 25 tok/s is the important number.

FHILY👑

39,811 Aufrufe • vor 23 Tagen