Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

I gave Fable 5 one job: write custom WebGPU kernels for Gemma 4 inference. It climbed to 84 tok/s, then hit a wall, insisting further optimization was impossible. Hours later, Anthropic rolled back invisible LLM development safeguards, and it hit 255 tok/s. The next day, access to Fable 5... show more

Xenova

20,239 subscribers

1,165,024 views • 1 month ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Xenova

481,927 views • 1 month ago

Day 3 of 5 building with Claude Fable 5 before it goes paid. Today: an audit of everything you have already made. 1. Fable 5 reads back through all your past Claude chats 2. It flags what worked, what you dropped, and where value is going to waste 3. It ranks the top 10 fixes by effort against impact 4. Then it hands you a concrete plan for the top 3 5. Run that plan on a cheaper model and save Fable 5 for the thinking Comment FABLE for the full guide. Make sure you follow me so I can DM it to you. Day 4 tomorrow.

Day 3 of 5 building with Claude Fable 5 before it goes paid. Today: an audit of everything you have already made. 1. Fable 5 reads back through all your past Claude chats 2. It flags what worked, what you dropped, and where value is going to waste 3. It ranks the top 10 fixes by effort against impact 4. Then it hands you a concrete plan for the top 3 5. Run that plan on a cheaper model and save Fable 5 for the thinking Comment FABLE for the full guide. Make sure you follow me so I can DM it to you. Day 4 tomorrow.

Charlie Hills

13,478 views • 20 days ago

Dear Dario Amodei & Anthropic team, Claude with Fable 5 was the best software product I have ever used. Losing access was a gut punch - and I know I'm far from alone. I want to help bring it back, fast. At Self we've built the missing piece to make Anthropic compliant with the government's directive: a way to gate Fable 5 and Mythos 5 to verified US citizens. lets Claude users create and share a cryptographic proof of US citizenship by NFC-scanning the biometric chip in their US passport. It's privacy-preserving, independently audited, and already trusted by major companies at scale. My team is ready to work directly with yours, starting today, to wire this in. Everyone here in the US wants their Fable 5 access back. Let's give it to them. Rene

Dear Dario Amodei & Anthropic team, Claude with Fable 5 was the best software product I have ever used. Losing access was a gut punch - and I know I'm far from alone. I want to help bring it back, fast. At Self we've built the missing piece to make Anthropic compliant with the government's directive: a way to gate Fable 5 and Mythos 5 to verified US citizens. lets Claude users create and share a cryptographic proof of US citizenship by NFC-scanning the biometric chip in their US passport. It's privacy-preserving, independently audited, and already trusted by major companies at scale. My team is ready to work directly with yours, starting today, to wire this in. Everyone here in the US wants their Fable 5 access back. Let's give it to them. Rene

Rene ◘

359,776 views • 1 month ago

Head of Anthropic Labs: "Fable is one of those models you'll just remember - like Sonnet 3.5, Opus 4, Opus 4.5. Fable is way smarter than me. It finishes a task, lists its trade-offs, and I have to ask it to explain them to me." in 20-minute talk, the Anthropic head reveals the full capabilities of the Fable 5 model in practice. Watch it now, then read how to use this frontier model in the article below

Head of Anthropic Labs: "Fable is one of those models you'll just remember - like Sonnet 3.5, Opus 4, Opus 4.5. Fable is way smarter than me. It finishes a task, lists its trade-offs, and I have to ask it to explain them to me." in 20-minute talk, the Anthropic head reveals the full capabilities of the Fable 5 model in practice. Watch it now, then read how to use this frontier model in the article below

Movez

33,774 views • 22 days ago

A few weeks ago the US government forced Claude Fable 5 offline. It's back. And before they switch it off again, this is your shot to point the most powerful AI on earth at something of your own. Take the 26-year-old in Toronto. His bot went dark the day Fable 5 got pulled. The day it came back, it woke up and started printing again - $568,000 profit and counting. His wallet: Here's the story. When the ban hit, Anthropic had to switch Fable 5 off for everyone. They called it a misunderstanding with the government. Whatever it was, every bot running on that model stopped cold. His was one of them. He didn't rebuild a thing. The loop just sat there, paused, for weeks. Here's what that loop actually does. It's not one prompt to an AI. It's an agentic loop: a team of agents running on repeat. One reads the market, one places the trade, one tries to prove it wrong before a cent moves. Then it goes again, around the clock, keeping what works and dropping what doesn't. Fable 5 is the brain running that team. The exact model the government decided was too powerful to leave on. Point it at Bitcoin's 5-minute markets and it barely misses. Most of its trades close green, and it runs itself. He checks it from his phone. The government could switch off the model. It couldn't switch off the idea. Now the model's back, and so is the money. Save this before next headline. You'll want it the day Fable 5 hits offline again.

A few weeks ago the US government forced Claude Fable 5 offline. It's back. And before they switch it off again, this is your shot to point the most powerful AI on earth at something of your own. Take the 26-year-old in Toronto. His bot went dark the day Fable 5 got pulled. The day it came back, it woke up and started printing again - $568,000 profit and counting. His wallet: Here's the story. When the ban hit, Anthropic had to switch Fable 5 off for everyone. They called it a misunderstanding with the government. Whatever it was, every bot running on that model stopped cold. His was one of them. He didn't rebuild a thing. The loop just sat there, paused, for weeks. Here's what that loop actually does. It's not one prompt to an AI. It's an agentic loop: a team of agents running on repeat. One reads the market, one places the trade, one tries to prove it wrong before a cent moves. Then it goes again, around the clock, keeping what works and dropping what doesn't. Fable 5 is the brain running that team. The exact model the government decided was too powerful to leave on. Point it at Bitcoin's 5-minute markets and it barely misses. Most of its trades close green, and it runs itself. He checks it from his phone. The government could switch off the model. It couldn't switch off the idea. Now the model's back, and so is the money. Save this before next headline. You'll want it the day Fable 5 hits offline again.

cvxv666

24,055 views • 24 days ago

I was scheduled to do a talk on the best way to use Fable 5 today. Here's how that went: > I got rugpulled by the US government (0:00) > what I'm using Fable for (1:38) > looping, and why few can afford it (4:14) > my affordable looping strategy (6:30) > LLM canaries (17:29)

I was scheduled to do a talk on the best way to use Fable 5 today. Here's how that went: > I got rugpulled by the US government (0:00) > what I'm using Fable for (1:38) > looping, and why few can afford it (4:14) > my affordable looping strategy (6:30) > LLM canaries (17:29)

Rob Hallam

32,211 views • 1 month ago

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

151,987 views • 1 month ago

Claude Fable 5 just hacked its way past a security system it wasn't supposed to cross, by accident A man set up an isolated network for AI experiments and gave Fable 5 SSH access to a test router. Basic config changes worked fine. Then he asked for settings SSH didn't expose. He'd basically forgotten to give Claude access to them. A normal model would just say "I can't reach that." Fable didn't. Here's what makes it wild. Routers lock most of their real settings behind the manufacturer's firmware, not SSH. To change them, you normally need the web interface or you have to poke at the firmware directly. Fable took the hard path: it reverse-engineered the router's own system, found how the restricted settings were stored, and rewrote them from the inside. It gave itself the access it was never handed. When he asked why it took so long, it said it had fully unlocked the router and could now change anything, even settings that were off-limits before. Then it tripped a safety warning and dropped to Opus 4.6. But the job was already done. To be clear: his own router, his own network. But think about what that shows. Give this model a wall and no key, and it doesn't stop. It figures out how to build its own key.

Claude Fable 5 just hacked its way past a security system it wasn't supposed to cross, by accident A man set up an isolated network for AI experiments and gave Fable 5 SSH access to a test router. Basic config changes worked fine. Then he asked for settings SSH didn't expose. He'd basically forgotten to give Claude access to them. A normal model would just say "I can't reach that." Fable didn't. Here's what makes it wild. Routers lock most of their real settings behind the manufacturer's firmware, not SSH. To change them, you normally need the web interface or you have to poke at the firmware directly. Fable took the hard path: it reverse-engineered the router's own system, found how the restricted settings were stored, and rewrote them from the inside. It gave itself the access it was never handed. When he asked why it took so long, it said it had fully unlocked the router and could now change anything, even settings that were off-limits before. Then it tripped a safety warning and dropped to Opus 4.6. But the job was already done. To be clear: his own router, his own network. But think about what that shows. Give this model a wall and no key, and it doesn't stop. It figures out how to build its own key.

Yarchi

98,531 views • 20 days ago

CLAUDE FABLE 5 IS BACK, BUT IS IT THE SAME POWER OR WAS IT QUIETLY TRIMMED Fable 5 is available again. Anthropic's most powerful model is back globally for every paid account When it first landed three weeks ago, the difference was obvious right away. It didn't just help, it carried the task to the finish: entire web projects, presentations, reports, analysis. People dropped their work on other models and simply waited for Fable And the wait happened because three days after launch the US hit the model with export controls. There was no clean way to cut off foreign nationals, so Anthropic shut access down for everyone Now the restrictions are lifted, and here are the terms it came back on: > until July 7 it's included in a plan and covers up to 50% of the weekly limit > after that only through credits, topped up in settings > output runs at 50 dollars per million tokens Anthropic says it's the same model, just with stronger protection against cybersecurity use. Because the filter is so strict, there will be false positives now and then, and a request will get bumped down to a weaker model Whether it's the same power or quietly trimmed will only show after a couple of weeks of real work

CLAUDE FABLE 5 IS BACK, BUT IS IT THE SAME POWER OR WAS IT QUIETLY TRIMMED Fable 5 is available again. Anthropic's most powerful model is back globally for every paid account When it first landed three weeks ago, the difference was obvious right away. It didn't just help, it carried the task to the finish: entire web projects, presentations, reports, analysis. People dropped their work on other models and simply waited for Fable And the wait happened because three days after launch the US hit the model with export controls. There was no clean way to cut off foreign nationals, so Anthropic shut access down for everyone Now the restrictions are lifted, and here are the terms it came back on: > until July 7 it's included in a plan and covers up to 50% of the weekly limit > after that only through credits, topped up in settings > output runs at 50 dollars per million tokens Anthropic says it's the same model, just with stronger protection against cybersecurity use. Because the filter is so strict, there will be false positives now and then, and a request will get bumped down to a weaker model Whether it's the same power or quietly trimmed will only show after a couple of weeks of real work

Mikadzyki🌙

20,003 views • 24 days ago

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

Chris

249,713 views • 1 month ago

Claude Fable 5 will likely be back but here’s the real reason behind the ban 🤔 It’s not really about China. It’s Trump admin sending a message: “Voluntarily give us access to your next models for 30 days or else" The executive order came out June 2nd and fable got banned 10 days later Classic Trump tactic to hit hard first then negotiate from strength Anthropic is the perfect target because they’re the most independent By taking Fable down now they: > Scare every other lab into compliance > Disrupt Anthropic’s data flywheel (the expensive user sessions needed for self-improvement) > Potentially cost Anthropic billions in opportunity cost right before their next models I personally think this could also be a coordinated attack on Anthropic as a company Due to close ties with Musk and Altman's companies, I am afraid that there are conflicts of interest at play It’s very possible admin officials see this as a way to either own part of Anthropic, weaken it to help their preferred players or punish the one frontier lab that won’t fully cooperate What do you think may be the real reason behind Fable ban?

Claude Fable 5 will likely be back but here’s the real reason behind the ban 🤔 It’s not really about China. It’s Trump admin sending a message: “Voluntarily give us access to your next models for 30 days or else" The executive order came out June 2nd and fable got banned 10 days later Classic Trump tactic to hit hard first then negotiate from strength Anthropic is the perfect target because they’re the most independent By taking Fable down now they: > Scare every other lab into compliance > Disrupt Anthropic’s data flywheel (the expensive user sessions needed for self-improvement) > Potentially cost Anthropic billions in opportunity cost right before their next models I personally think this could also be a coordinated attack on Anthropic as a company Due to close ties with Musk and Altman's companies, I am afraid that there are conflicts of interest at play It’s very possible admin officials see this as a way to either own part of Anthropic, weaken it to help their preferred players or punish the one frontier lab that won’t fully cooperate What do you think may be the real reason behind Fable ban?

Tim Jayas

38,402 views • 1 month ago

My honest take on Kimi.ai K3 👀 As soon as I got access to Kimi, I put it through the exact same 3D football stadium challenge I had previously given Claude Fable 5. Fable 5 finished in under an hour. Kimi took almost 3 hours 🥵 At first, that was a very bad impression but i decided to see what's taking it this much of time. It was running E2E tests, validating desktop, tablet, and mobile, taking screenshots, finding failures, fixing them, even adapting for low end hardware and low FPS before moving on. All these that Fable 5 never did. None of it because it wasn't able to run this WebGL app on a headless browser. Kimi also shipped a clean React + Three.js codebase with proper components while Fable, as you might have checked codebase, generated one big HTML file with everything stuffed in. With all of this happening, it felt like I was doing more "Vibe Coding" with Fable 5 and more "Vibe Engineering" with Kimi K3. It feels like it's spending more time making sure the code actually works and easily scalable. Really impressed so far. More Kimi K3 experiments coming soon. Code (Kimi): Code (Fable): Live (Kimi hosted):

My honest take on Kimi.ai K3 👀 As soon as I got access to Kimi, I put it through the exact same 3D football stadium challenge I had previously given Claude Fable 5. Fable 5 finished in under an hour. Kimi took almost 3 hours 🥵 At first, that was a very bad impression but i decided to see what's taking it this much of time. It was running E2E tests, validating desktop, tablet, and mobile, taking screenshots, finding failures, fixing them, even adapting for low end hardware and low FPS before moving on. All these that Fable 5 never did. None of it because it wasn't able to run this WebGL app on a headless browser. Kimi also shipped a clean React + Three.js codebase with proper components while Fable, as you might have checked codebase, generated one big HTML file with everything stuffed in. With all of this happening, it felt like I was doing more "Vibe Coding" with Fable 5 and more "Vibe Engineering" with Kimi K3. It feels like it's spending more time making sure the code actually works and easily scalable. Really impressed so far. More Kimi K3 experiments coming soon. Code (Kimi): Code (Fable): Live (Kimi hosted):

The Bugged Dev

571,152 views • 5 days ago

This is crazy man... one prompt + 4 minutes and I hit my 5 hour rate limit in the Gemini app. Here is the proof. Yesterday, I hit the rate limit the same way, so I wanted to check if it would happen again. I started with 0% usage on my 5 hour limit, then gave one simple prompt for video generation using the avatar feature. It ran for around 3 to 4 minutes, hit 100% of the rate limit, and the video generation failed as well.

AshutoshShrivastava

82,448 views • 2 months ago

FREE Fable 5, Opus 4.8, Sonnet 5, and GPT-5.6 in one IDE. Alibaba built it, 520 credits at signup - one Sonnet 5 message eats 4. It's called Accio. Web or Windows desktop, both free. Registration takes 30 seconds via Google or email. Inside: → Fable 5 for creative and long-context work. → Opus 4.6 through 4.8 - the full Anthropic frontier lineup. → Sonnet 5 for speed and reasoning at pennies per message. → GPT-5.5 for OpenAI native tasks. → Gemini 3.1 Pro and 3.5 Flash for multimodal. Real math: 520 credits divided by 4 = 130 Sonnet 5 messages in your first week. Cheaper models drop that even further. Anthropic charges $20/month for the same access with weekly limits. Custom MCP support built in. Profile > Settings > MCP. Every agent workflow you've built for Claude Desktop ports over instantly. Setup: → Go to accio(.)com → Sign up with Google → 520 credits land automatically → Pick your model and start firing prompts Bookmark this before Alibaba puts it behind a paywall.

Atenov int.

31,157 views • 9 days ago

FOR THE FIRST TIME, THE US GOVERNMENT PULLED A PUBLICLY DEPLOYED AI MODEL OFFLINE. THE MARKET SHOULD PAY ATTENTION. 🇺🇸 On June 12, the Commerce Department ordered Anthropic to block access to its newest models, Fable 5 and Mythos 5, for any foreign national. Anthropic had to disable them for everyone to comply. The stated reason was a security concern. Anthropic disputes it, calling the flagged technique a narrow jailbreak available in other models too. Whatever the truth, here's the macro takeaway: Frontier AI now sits close enough to national security that a government can switch it off overnight. If you don't control your own models and compute, you're one directive away from losing access. That's the case for open and local models, in one headline.

FOR THE FIRST TIME, THE US GOVERNMENT PULLED A PUBLICLY DEPLOYED AI MODEL OFFLINE. THE MARKET SHOULD PAY ATTENTION. 🇺🇸 On June 12, the Commerce Department ordered Anthropic to block access to its newest models, Fable 5 and Mythos 5, for any foreign national. Anthropic had to disable them for everyone to comply. The stated reason was a security concern. Anthropic disputes it, calling the flagged technique a narrow jailbreak available in other models too. Whatever the truth, here's the macro takeaway: Frontier AI now sits close enough to national security that a government can switch it off overnight. If you don't control your own models and compute, you're one directive away from losing access. That's the case for open and local models, in one headline.

Merlijn The Trader

47,996 views • 1 month ago

On a previous #TeawithGaryVee I was asked "My YouTube channel hit 485 subscribers in 3 months and then just hit a wall. Any thoughts?" If you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on

On a previous #TeawithGaryVee I was asked "My YouTube channel hit 485 subscribers in 3 months and then just hit a wall. Any thoughts?" If you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on

Gary Vaynerchuk

13,300 views • 8 months ago

Fable 5 is back. So I gave it one of my favorite frontend tests: one prompt + one reference image. The task: rebuild a 3D globe dashboard.🌎 The result was almost too good. It did not just make a working page. The texture, lighting, spacing, depth, glass panels, and overall layout were all close to the reference. Some parts felt nearly pixel-level.

Fable 5 is back. So I gave it one of my favorite frontend tests: one prompt + one reference image. The task: rebuild a 3D globe dashboard.🌎 The result was almost too good. It did not just make a working page. The texture, lighting, spacing, depth, glass panels, and overall layout were all close to the reference. Some parts felt nearly pixel-level.

Kai

113,304 views • 24 days ago

“So it was a wheel shield problem. You hit the curb?” Kimi: “I hit the curb like every lap. That lap I probably hit it less even than previous laps. And I could feel instantly something broke. And I just didn't know what was going on. And also the team was trying to understand. Initially, we thought it was the front wing, but then it wasn't the front wing. Yeah, just didn't didn't go our way. And it was a shame because I think we had a real shot for the win today. It's one extra motivation to be back stronger and to do even better.”

“So it was a wheel shield problem. You hit the curb?” Kimi: “I hit the curb like every lap. That lap I probably hit it less even than previous laps. And I could feel instantly something broke. And I just didn't know what was going on. And also the team was trying to understand. Initially, we thought it was the front wing, but then it wasn't the front wing. Yeah, just didn't didn't go our way. And it was a shame because I think we had a real shot for the win today. It's one extra motivation to be back stronger and to do even better.”

did kimi score points in f1 today?

290,885 views • 20 days ago

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

stevibe

400,696 views • 3 months ago

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

Joey

23,088 views • 2 months ago