Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Meet Wally. Our inference stack for open frontier models, built to be the fastest place to run them. Performance snapshot: GLM-5.3 Flash: 380 tok/s. GLM-5.3 Max: 790 tok/s Qwen3.8-27B: 485 tok/s DeepSeek-V4.1 Flash: 615 tok/s 1/7

RunAnywhere

2,123 subscribers

61,190 views • 4 days ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

60 Comments

RunAnywhere4 days ago

Benchmarks don't always tell the reality. So we ran a real prompt, same OpenCode harness, GLM-5.3 Flash on Wally, @nebiusai , @FireworksAI_HQ and @Zai_org , all building at once. Wally delivers 43% more throughput than Nebius, 3.1× Fireworks, and 4.5× Z ai. 2/7

RunAnywhere4 days ago

Drop Wally into the harness you already use. Live right now for: ->Claude Code ->Claude Desktop ->OpenCode ->DeepSeek Harness ->Hermes ->OpenClaw Blazing fast open-model inference, first token in under 300 ms. $5 free, Drop this in your favorite coding agent to get started: "set up 3/7

RunAnywhere4 days ago

We raced Wally, Nebius, Fireworks and Z ai 3 times on a real prompt. Here are the results: 4/7

RunAnywhere4 days ago

GLM-5.3 Flash across every provider on Artificial Analysis, Sep 2026. Nebius 313 tok/s. Fireworks 185 tok/s. The model maker's own API: 72 tok/s. Wally: 380 tok/s, measured the same way. 5/7

RunAnywhere4 days ago

Every model on wally runs at state-of-the-art speed. That matters the most for realtime apps:Voice pipelines with an LLM in the loop, coding agents, browser automation, live copilots. On our GPUs, or yours(BYOC/on-prem) 6/7

RunAnywhere4 days ago

Get Started: Drop this into your favorite coding agent: "set up or Curl : curl -fsSL | sh Windows: irm | iex wally login wally claude-code -m glm-5.3-flash $5 free on signup, no card. Standard per-token after. 7/7

Eric Liu4 days ago

insane numbers

Deesha Tech4 days ago

I couldn't use your console, nothing works. Cannot create keys, cannot run playground, console crashes most of the time and have to refresh. Seems sloppy launch

RunAnywhere4 days ago

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Deesha Tech4 days ago

Great thanks, its working now

David Gu4 days ago

Congrats on the launch!!

RunAnywhere4 days ago

🫱🏻‍🫲🏽🙌🏽

Yohei4 days ago

installing!

RunAnywhere4 days ago

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

MrOzi4 days ago

Those numbers are fast enough to make "Where's Wally?" a rhetorical question.

Zachary Yu4 days ago

lfg!

Landseer Enga4 days ago

AWESOME!

RunAnywhere4 days ago

Wally 🫱🏻‍🫲🏽 @tryrevyl ? 👀

Ishaan4 days ago

Congrats on the launch 🚀

RunAnywhere4 days ago

🫱🏻‍🫲🏽

RAZA | AI EXPLORER4 days ago

Those inference speeds are seriously impressive.

RunAnywhere4 days ago

⚡️⚡️

Luis Costa4 days ago

Lfg team!! 🔥🔥

RunAnywhere4 days ago

🫱🏻‍🫲🏽

Tony Gao4 days ago

Super cool stuff!

RunAnywhere4 days ago

🙏🫱🏻‍🫲🏽

Owen Botkin4 days ago

so cool

TensorQuay4 days ago

The stop button deserves some love too. Wally's docs say Esc in Claude Code cancels the hosted request; during prefill it waits for the first token's request ID. Handy when an agent confidently starts solving the wrong problem.

Shivay Lamba4 days ago

huge congrats on the launch!

RunAnywhere4 days ago

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

Beshoy Saad4 days ago

where are intelligence numbers? there's gotta be insane fall off...

Nico4 days ago

nice video

RunAnywhere4 days ago

Made by GLM 5.3 flash using Wally :)

Aakash Mahalingam4 days ago

LFG 🚀🚀

RunAnywhere4 days ago

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

Gerard Smit4 days ago

Where are the prices? It says it's in the console, but the console doesn't show models? Making a key also doesn't work; too much traffic?😅

RunAnywhere4 days ago

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Gerard Smit4 days ago

Doesn't seem to be working yet. In EU (Netherlands) I'm getting infinite load on AJAX calls to "

Gerard Smit4 days ago

Seems to be working now 😄

Silen Naihin4 days ago

LFG

Piyush Yadav | Top1Board | PassReady4 days ago

These numbers are absurd—getting nearly 500 tok/s on a 27B parameter model like Qwen completely changes the architecture for real-time applications. Are these benchmarks on Apple Silicon unified memory or a cloud GPU cluster? Need to know before I hook it into My App

MJ4 days ago

No pricing and so broken

RunAnywhere4 days ago

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Peasant Smith4 days ago

But did you find... You know?

Sadok4 days ago

those deepseek speeds are getting ridiculous

Trunk Monkey4 days ago

Seems pretty decent but I got rate limited just doing some really basic stuff. Did resume a previous session but still.

RunAnywhere4 days ago

Thank you for the feedback! We’re working hard to support the demand

Trunk Monkey4 days ago

Seems like a cool service but just hit that again. Likely the agent got stale and just sending its cache + 1 tiny command put me over the limit.

Shadow4 days ago

flash isn't a speed tier, it's a compression trick

Velsinha4 days ago

Will you offer plans, or just pay-per-use?

RunAnywhere4 days ago

More info soon. Stay tuned!

vikas4 days ago

guys when are you adding deepseek v4.1 flash? really waiting for it

BayesChat4 days ago

The real-prompt comparison is more useful than a synthetic leaderboard. Same harness and workload make the throughput gap much easier to understand.

Viswesh N G4 days ago

Congrats this is awesome!!🔥

cetusian4 days ago

@yoheinakajima bro that’s wall-e

RunAnywhere4 days ago

@yoheinakajima We like Wally!

sss4 days ago

On what hardware

Nilan Saha4 days ago

Whats the quantization? No point in a lobotomized model thats faster

_alphashark_4 days ago

at 790 tok/s, a 100-token answer takes about 127 ms to decode. throughput alone won't show whether time-to-first-token or queueing dominates the inference stack.

EDDY VU4 days ago

790 tok/s on GLM-5.3 Max is insane, how much of that speedup comes from custom kernels versus speculative decoding?

Related Videos

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,628 views • 29 days ago

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

Mia

16,556 views • 18 days ago

Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → RTX 5090 desktop @ 22-25 tok/s GLM-5.2 753B → RTX PRO 6000 workstation @ 15 tok/s Run your claude code or codex now with frontier model for $0 Meet FreeToken 🧵

Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → RTX 5090 desktop @ 22-25 tok/s GLM-5.2 753B → RTX PRO 6000 workstation @ 15 tok/s Run your claude code or codex now with frontier model for $0 Meet FreeToken 🧵

Shuo Yang

1,098,022 views • 1 month ago

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

Awni Hannun

141,194 views • 8 months ago

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Killy

30,081 views • 22 days ago

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

Abdur Rahim

107,726 views • 11 days ago

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

Mia

23,835 views • 10 days ago

GLM 5.3 on 8 x RTX Blackwell 6000's @ 120 tok/s with OMP. No skills. No plugins. Local Qwen 3.8 27B for vision 👀 Great at making neat little things to look at, like this ship in a bottle! One-shot prompt below 👇

GLM 5.3 on 8 x RTX Blackwell 6000's @ 120 tok/s with OMP. No skills. No plugins. Local Qwen 3.8 27B for vision 👀 Great at making neat little things to look at, like this ship in a bottle! One-shot prompt below 👇

wick

71,682 views • 25 days ago

$0.049. DeepSeek V4.1 Flash used ego (lite) to shop Amazon for a MacBook desk setup, all under $100. V4.1 Flash was pushing 295 tok/s. ego (lite) kept the clicks moving. The cart had a productive five minutes.

$0.049. DeepSeek V4.1 Flash used ego (lite) to shop Amazon for a MacBook desk setup, all under $100. V4.1 Flash was pushing 295 tok/s. ego (lite) kept the clicks moving. The cart had a productive five minutes.

ego

33,396 views • 17 days ago

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

0xSero

74,772 views • 5 months ago

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

bstn 👁️

23,120 views • 5 months ago

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

atomic.chat

32,515 views • 24 days ago

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

158,079 views • 2 months ago

Qwen3.8-Flash-Next just got a massive speed boost on my old hardware. This guy started with roughly 11 tok/s. After tuning the inference stack, I’m now getting around 24–28 tok/s. That’s roughly a 2× speedup without changing the GPU. The setup: → RTX 3090 24GB → 64GB DDR4 system RAM → Qwen3.8-Flash-Next → Cafe-llama.cpp The interesting part is that Cafe-llama.cpp isn’t just another llama.cpp fork. The optimizations are making a real difference for this particular architecture, especially when you’re dealing with a model that relies heavily on system memory and offloading. Going from 11 tok/s to nearly 30 tok/s completely changes how practical the model feels for local use. And now I’m curious what happens with DDR5. If memory bandwidth becomes another bottleneck, a faster RAM subsystem could squeeze even more performance out of the same 3090. This is why local AI optimization is getting so interesting. Sometimes you don’t need a new GPU. You just need a better inference stack. 11 tok/s → 24–28 tok/s.

Qwen3.8-Flash-Next just got a massive speed boost on my old hardware. This guy started with roughly 11 tok/s. After tuning the inference stack, I’m now getting around 24–28 tok/s. That’s roughly a 2× speedup without changing the GPU. The setup: → RTX 3090 24GB → 64GB DDR4 system RAM → Qwen3.8-Flash-Next → Cafe-llama.cpp The interesting part is that Cafe-llama.cpp isn’t just another llama.cpp fork. The optimizations are making a real difference for this particular architecture, especially when you’re dealing with a model that relies heavily on system memory and offloading. Going from 11 tok/s to nearly 30 tok/s completely changes how practical the model feels for local use. And now I’m curious what happens with DDR5. If memory bandwidth becomes another bottleneck, a faster RAM subsystem could squeeze even more performance out of the same 3090. This is why local AI optimization is getting so interesting. Sometimes you don’t need a new GPU. You just need a better inference stack. 11 tok/s → 24–28 tok/s.

FHILY👑

23,243 views • 26 days ago

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

SGLang

99,353 views • 29 days ago

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro ⚡ Meet Inco Splash: our open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents.

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro ⚡ Meet Inco Splash: our open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents.

Inco AI

938,320 views • 7 days ago

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

Baseten

23,114 views • 29 days ago

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Zhijian Liu

92,333 views • 7 days ago

deepseek v4.1 flash is now 100% free 😳 a new coding agent is using ads to keep the whole thing free. > DeepSeek V4.1 Flash > GPT 6 Luna > GLM 5.3 Flash > More model How to get start → go to [ → sign up with GitHub → generate your API key these models be used for six hours per day.

deepseek v4.1 flash is now 100% free 😳 a new coding agent is using ads to keep the whole thing free. > DeepSeek V4.1 Flash > GPT 6 Luna > GLM 5.3 Flash > More model How to get start → go to [ → sign up with GitHub → generate your API key these models be used for six hours per day.

BARIK

23,850 views • 10 days ago

125B on a single RTX 4090 at 25 tok/s is actually insane Qwen3.8-Flash-Next + MTP is hitting: 80K context 25.35 tok/s decode 471 tok/s prefill On one consumer GPU. Yes, it’s an MoE, so you’re not running all 125B parameters for every token. But still, getting a model this large to run this fast on a 4090 is wild. MTP is doing a lot of the heavy lifting here too. Instead of generating one token at a time, it can speculate on multiple tokens and verify them together. And 25 tok/s is the important number.

125B on a single RTX 4090 at 25 tok/s is actually insane Qwen3.8-Flash-Next + MTP is hitting: 80K context 25.35 tok/s decode 471 tok/s prefill On one consumer GPU. Yes, it’s an MoE, so you’re not running all 125B parameters for every token. But still, getting a model this large to run this fast on a 4090 is wild. MTP is doing a lot of the heavy lifting here too. Instead of generating one token at a time, it can speculate on multiple tokens and verify them together. And 25 tok/s is the important number.

FHILY👑

39,811 views • 23 days ago