Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Meet Wally. Our inference stack for open frontier models, built to be the fastest place to run them. Performance snapshot: GLM-5.3 Flash: 380 tok/s. GLM-5.3 Max: 790 tok/s Qwen3.8-27B: 485 tok/s DeepSeek-V4.1 Flash: 615 tok/s 1/7

RunAnywhere

2,123 subscribers

61,190 görüntüleme • 4 gün önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

60 Yorum

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Benchmarks don't always tell the reality. So we ran a real prompt, same OpenCode harness, GLM-5.3 Flash on Wally, @nebiusai , @FireworksAI_HQ and @Zai_org , all building at once. Wally delivers 43% more throughput than Nebius, 3.1× Fireworks, and 4.5× Z ai. 2/7

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Drop Wally into the harness you already use. Live right now for: ->Claude Code ->Claude Desktop ->OpenCode ->DeepSeek Harness ->Hermes ->OpenClaw Blazing fast open-model inference, first token in under 300 ms. $5 free, Drop this in your favorite coding agent to get started: "set up 3/7

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

We raced Wally, Nebius, Fireworks and Z ai 3 times on a real prompt. Here are the results: 4/7

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

GLM-5.3 Flash across every provider on Artificial Analysis, Sep 2026. Nebius 313 tok/s. Fireworks 185 tok/s. The model maker's own API: 72 tok/s. Wally: 380 tok/s, measured the same way. 5/7

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Every model on wally runs at state-of-the-art speed. That matters the most for realtime apps:Voice pipelines with an LLM in the loop, coding agents, browser automation, live copilots. On our GPUs, or yours(BYOC/on-prem) 6/7

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Get Started: Drop this into your favorite coding agent: "set up or Curl : curl -fsSL | sh Windows: irm | iex wally login wally claude-code -m glm-5.3-flash $5 free on signup, no card. Standard per-token after. 7/7

Eric Liu profil fotoğrafı

Eric Liu4 gün önce

insane numbers

Deesha Tech profil fotoğrafı

Deesha Tech4 gün önce

I couldn't use your console, nothing works. Cannot create keys, cannot run playground, console crashes most of the time and have to refresh. Seems sloppy launch

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Deesha Tech profil fotoğrafı

Deesha Tech4 gün önce

Great thanks, its working now

David Gu profil fotoğrafı

David Gu4 gün önce

Congrats on the launch!!

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

🫱🏻‍🫲🏽🙌🏽

Yohei profil fotoğrafı

Yohei4 gün önce

installing!

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

MrOzi profil fotoğrafı

MrOzi4 gün önce

Those numbers are fast enough to make "Where's Wally?" a rhetorical question.

Zachary Yu profil fotoğrafı

Zachary Yu4 gün önce

lfg!

Landseer Enga profil fotoğrafı

Landseer Enga4 gün önce

AWESOME!

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Wally 🫱🏻‍🫲🏽 @tryrevyl ? 👀

Ishaan profil fotoğrafı

Ishaan4 gün önce

Congrats on the launch 🚀

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

🫱🏻‍🫲🏽

RAZA | AI EXPLORER profil fotoğrafı

RAZA | AI EXPLORER4 gün önce

Those inference speeds are seriously impressive.

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

⚡️⚡️

Luis Costa profil fotoğrafı

Luis Costa4 gün önce

Lfg team!! 🔥🔥

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

🫱🏻‍🫲🏽

Tony Gao profil fotoğrafı

Tony Gao4 gün önce

Super cool stuff!

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

🙏🫱🏻‍🫲🏽

Owen Botkin profil fotoğrafı

Owen Botkin4 gün önce

so cool

TensorQuay profil fotoğrafı

TensorQuay4 gün önce

The stop button deserves some love too. Wally's docs say Esc in Claude Code cancels the hosted request; during prefill it waits for the first token's request ID. Handy when an agent confidently starts solving the wrong problem.

Shivay Lamba profil fotoğrafı

Shivay Lamba4 gün önce

huge congrats on the launch!

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

Beshoy Saad profil fotoğrafı

Beshoy Saad4 gün önce

where are intelligence numbers? there's gotta be insane fall off...

Nico profil fotoğrafı

Nico4 gün önce

nice video

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Made by GLM 5.3 flash using Wally :)

Aakash Mahalingam profil fotoğrafı

Aakash Mahalingam4 gün önce

LFG 🚀🚀

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

🫱🏻‍🫲🏽🫱🏻‍🫲🏽

Gerard Smit profil fotoğrafı

Gerard Smit4 gün önce

Where are the prices? It says it's in the console, but the console doesn't show models? Making a key also doesn't work; too much traffic?😅

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Gerard Smit profil fotoğrafı

Gerard Smit4 gün önce

Doesn't seem to be working yet. In EU (Netherlands) I'm getting infinite load on AJAX calls to "

Gerard Smit profil fotoğrafı

Gerard Smit4 gün önce

Seems to be working now 😄

Silen Naihin profil fotoğrafı

Silen Naihin4 gün önce

LFG

Piyush Yadav | Top1Board | PassReady profil fotoğrafı

Piyush Yadav | Top1Board | PassReady4 gün önce

These numbers are absurd—getting nearly 500 tok/s on a 27B parameter model like Qwen completely changes the architecture for real-time applications. Are these benchmarks on Apple Silicon unified memory or a cloud GPU cluster? Need to know before I hook it into My App

MJ profil fotoğrafı

MJ4 gün önce

No pricing and so broken

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Hey, sorry for the inconvenience. The console may have been unavailable for a short period due to an unexpected surge in traffic. Everything is back up and running now.

Peasant Smith profil fotoğrafı

Peasant Smith4 gün önce

But did you find... You know?

Sadok profil fotoğrafı

Sadok4 gün önce

those deepseek speeds are getting ridiculous

Trunk Monkey profil fotoğrafı

Trunk Monkey4 gün önce

Seems pretty decent but I got rate limited just doing some really basic stuff. Did resume a previous session but still.

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

Thank you for the feedback! We’re working hard to support the demand

Trunk Monkey profil fotoğrafı

Trunk Monkey4 gün önce

Seems like a cool service but just hit that again. Likely the agent got stale and just sending its cache + 1 tiny command put me over the limit.

Shadow profil fotoğrafı

Shadow4 gün önce

flash isn't a speed tier, it's a compression trick

Velsinha profil fotoğrafı

Velsinha4 gün önce

Will you offer plans, or just pay-per-use?

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

More info soon. Stay tuned!

vikas profil fotoğrafı

vikas4 gün önce

guys when are you adding deepseek v4.1 flash? really waiting for it

BayesChat profil fotoğrafı

BayesChat4 gün önce

The real-prompt comparison is more useful than a synthetic leaderboard. Same harness and workload make the throughput gap much easier to understand.

Viswesh N G profil fotoğrafı

Viswesh N G4 gün önce

Congrats this is awesome!!🔥

cetusian profil fotoğrafı

cetusian4 gün önce

@yoheinakajima bro that’s wall-e

RunAnywhere profil fotoğrafı

RunAnywhere4 gün önce

@yoheinakajima We like Wally!

sss profil fotoğrafı

sss4 gün önce

On what hardware

Nilan Saha profil fotoğrafı

Nilan Saha4 gün önce

Whats the quantization? No point in a lobotomized model thats faster

_alphashark_ profil fotoğrafı

_alphashark_4 gün önce

at 790 tok/s, a 100-token answer takes about 127 ms to decode. throughput alone won't show whether time-to-first-token or queueing dominates the inference stack.

EDDY VU profil fotoğrafı

EDDY VU4 gün önce

790 tok/s on GLM-5.3 Max is insane, how much of that speedup comes from custom kernels versus speculative decoding?

Benzer Videolar

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

GLM-5.3-Flash running at up to 300 tok/s single stream, and 1000 tok/s with 8 concurrent sessions. On average it's working at 150 tok/s per stream. Phenomenal model, really something special. On 2x Sparks we get 60 tok/s and 4x 100 tok/s 300B params is the sweet spot.

0xSero

30,628 görüntüleme • 29 gün önce

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

GLM 5.3 Flash on 2x DGX Sparks Look at the tok/s at the top :) This is prose. Releasing soon

Mia

16,556 görüntüleme • 18 gün önce

Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → RTX 5090 desktop @ 22-25 tok/s GLM-5.2 753B → RTX PRO 6000 workstation @ 15 tok/s Run your claude code or codex now with frontier model for $0 Meet FreeToken 🧵

Your gaming PC can now serve frontier models at interactive speed using official checkpoints without extreme quantization! Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → RTX 5090 desktop @ 22-25 tok/s GLM-5.2 753B → RTX PRO 6000 workstation @ 15 tok/s Run your claude code or codex now with frontier model for $0 Meet FreeToken 🧵

Shuo Yang

1,098,022 görüntüleme • 1 ay önce

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

Awni Hannun

141,194 görüntüleme • 8 ay önce

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Warning: Do not stare too long, trust me GLM 5.3 Flash TP=4 5718 tok/s 512 Concurrent Users 6610 tok/s C=768 Engine Decode Tried c=1024, don't fit Wall of c=256 each generating unique code (couldn't fit more on browser, which is probably good thing)

Killy

30,081 görüntüleme • 22 gün önce

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

Abdur Rahim

107,726 görüntüleme • 11 gün önce

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

DeepSeek v4.1 Flash vs GLM 5.3 Flash I asked both models to recreate the Psychonauts 2 and Metroid Prime game menus as closely as possible to the originals. Attached are full-sized videos from both models for both games. All were done in a single shot, both at Max thinking, same harness. Tokens used: DeepSeek v4.1 Flash - Psychonauts 2: 412k - Metroid Prime: 466k GLM 5.3 Flash - Psychonauts 2: 134k - Metroid Prime: 111k That's between 3-4x MORE used by DS4.1F. Who do you think won? IMO GLM 5.3 Flash won by FAR on both.

Mia

23,835 görüntüleme • 10 gün önce

GLM 5.3 on 8 x RTX Blackwell 6000's @ 120 tok/s with OMP. No skills. No plugins. Local Qwen 3.8 27B for vision 👀 Great at making neat little things to look at, like this ship in a bottle! One-shot prompt below 👇

GLM 5.3 on 8 x RTX Blackwell 6000's @ 120 tok/s with OMP. No skills. No plugins. Local Qwen 3.8 27B for vision 👀 Great at making neat little things to look at, like this ship in a bottle! One-shot prompt below 👇

wick

71,682 görüntüleme • 25 gün önce

$0.049. DeepSeek V4.1 Flash used ego (lite) to shop Amazon for a MacBook desk setup, all under $100. V4.1 Flash was pushing 295 tok/s. ego (lite) kept the clicks moving. The cart had a productive five minutes.

$0.049. DeepSeek V4.1 Flash used ego (lite) to shop Amazon for a MacBook desk setup, all under $100. V4.1 Flash was pushing 295 tok/s. ego (lite) kept the clicks moving. The cart had a productive five minutes.

ego

33,396 görüntüleme • 17 gün önce

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

0xSero

74,772 görüntüleme • 5 ay önce

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

bstn 👁️

23,120 görüntüleme • 5 ay önce

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

GLM 5.3 Flash running locally makes a Minecraft black hole mod We ran GLM 5.3 Flash Q4 locally on a rented 4x RTX PRO 6000 box and asked it to make a lightning gun mod for the real game using the Fabric API Output: -7.6M tokens -Time: ~9 hours -Decode: ~96 tok/s The mod adds a black hole rifle, when shot it spawns a black hole that pulls in blocks, with light rings that shrink all the way into the singularity, then collapses into an explosion crater that wipes out several chunks Run GLM 5.3 Flash locally in

atomic.chat

32,515 görüntüleme • 24 gün önce

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

158,079 görüntüleme • 2 ay önce

Qwen3.8-Flash-Next just got a massive speed boost on my old hardware. This guy started with roughly 11 tok/s. After tuning the inference stack, I’m now getting around 24–28 tok/s. That’s roughly a 2× speedup without changing the GPU. The setup: → RTX 3090 24GB → 64GB DDR4 system RAM → Qwen3.8-Flash-Next → Cafe-llama.cpp The interesting part is that Cafe-llama.cpp isn’t just another llama.cpp fork. The optimizations are making a real difference for this particular architecture, especially when you’re dealing with a model that relies heavily on system memory and offloading. Going from 11 tok/s to nearly 30 tok/s completely changes how practical the model feels for local use. And now I’m curious what happens with DDR5. If memory bandwidth becomes another bottleneck, a faster RAM subsystem could squeeze even more performance out of the same 3090. This is why local AI optimization is getting so interesting. Sometimes you don’t need a new GPU. You just need a better inference stack. 11 tok/s → 24–28 tok/s.

Qwen3.8-Flash-Next just got a massive speed boost on my old hardware. This guy started with roughly 11 tok/s. After tuning the inference stack, I’m now getting around 24–28 tok/s. That’s roughly a 2× speedup without changing the GPU. The setup: → RTX 3090 24GB → 64GB DDR4 system RAM → Qwen3.8-Flash-Next → Cafe-llama.cpp The interesting part is that Cafe-llama.cpp isn’t just another llama.cpp fork. The optimizations are making a real difference for this particular architecture, especially when you’re dealing with a model that relies heavily on system memory and offloading. Going from 11 tok/s to nearly 30 tok/s completely changes how practical the model feels for local use. And now I’m curious what happens with DDR5. If memory bandwidth becomes another bottleneck, a faster RAM subsystem could squeeze even more performance out of the same 3090. This is why local AI optimization is getting so interesting. Sometimes you don’t need a new GPU. You just need a better inference stack. 11 tok/s → 24–28 tok/s.

FHILY👑

23,243 görüntüleme • 26 gün önce

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

SGLang

99,353 görüntüleme • 29 gün önce

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro ⚡ Meet Inco Splash: our open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents.

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro ⚡ Meet Inco Splash: our open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents.

Inco AI

938,320 görüntüleme • 7 gün önce

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

Baseten

23,114 görüntüleme • 29 gün önce

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Zhijian Liu

92,333 görüntüleme • 7 gün önce

deepseek v4.1 flash is now 100% free 😳 a new coding agent is using ads to keep the whole thing free. > DeepSeek V4.1 Flash > GPT 6 Luna > GLM 5.3 Flash > More model How to get start → go to [ → sign up with GitHub → generate your API key these models be used for six hours per day.

deepseek v4.1 flash is now 100% free 😳 a new coding agent is using ads to keep the whole thing free. > DeepSeek V4.1 Flash > GPT 6 Luna > GLM 5.3 Flash > More model How to get start → go to [ → sign up with GitHub → generate your API key these models be used for six hours per day.

BARIK

23,850 görüntüleme • 10 gün önce

125B on a single RTX 4090 at 25 tok/s is actually insane Qwen3.8-Flash-Next + MTP is hitting: 80K context 25.35 tok/s decode 471 tok/s prefill On one consumer GPU. Yes, it’s an MoE, so you’re not running all 125B parameters for every token. But still, getting a model this large to run this fast on a 4090 is wild. MTP is doing a lot of the heavy lifting here too. Instead of generating one token at a time, it can speculate on multiple tokens and verify them together. And 25 tok/s is the important number.

125B on a single RTX 4090 at 25 tok/s is actually insane Qwen3.8-Flash-Next + MTP is hitting: 80K context 25.35 tok/s decode 471 tok/s prefill On one consumer GPU. Yes, it’s an MoE, so you’re not running all 125B parameters for every token. But still, getting a model this large to run this fast on a 4090 is wild. MTP is doing a lot of the heavy lifting here too. Instead of generating one token at a time, it can speculate on multiple tokens and verify them together. And 25 tok/s is the important number.

FHILY👑

39,811 görüntüleme • 23 gün önce