Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

I gave Fable 5 one job: write custom WebGPU kernels for Gemma 4 inference. It climbed to 84 tok/s, then hit a wall, insisting further optimization was impossible. Hours later, Anthropic rolled back invisible LLM development safeguards, and it hit 255 tok/s. The next day, access to Fable 5... show more

Xenova

19,742 subscribers

1,163,660 просмотров • 1 месяц назад •via X (Twitter)

Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Xenova

479,956 просмотров • 28 дней назад

Day 3 of 5 building with Claude Fable 5 before it goes paid. Today: an audit of everything you have already made. 1. Fable 5 reads back through all your past Claude chats 2. It flags what worked, what you dropped, and where value is going to waste 3. It ranks the top 10 fixes by effort against impact 4. Then it hands you a concrete plan for the top 3 5. Run that plan on a cheaper model and save Fable 5 for the thinking Comment FABLE for the full guide. Make sure you follow me so I can DM it to you. Day 4 tomorrow.

Day 3 of 5 building with Claude Fable 5 before it goes paid. Today: an audit of everything you have already made. 1. Fable 5 reads back through all your past Claude chats 2. It flags what worked, what you dropped, and where value is going to waste 3. It ranks the top 10 fixes by effort against impact 4. Then it hands you a concrete plan for the top 3 5. Run that plan on a cheaper model and save Fable 5 for the thinking Comment FABLE for the full guide. Make sure you follow me so I can DM it to you. Day 4 tomorrow.

Charlie Hills

13,478 просмотров • 10 дней назад

Dear Dario Amodei & Anthropic team, Claude with Fable 5 was the best software product I have ever used. Losing access was a gut punch - and I know I'm far from alone. I want to help bring it back, fast. At Self we've built the missing piece to make Anthropic compliant with the government's directive: a way to gate Fable 5 and Mythos 5 to verified US citizens. lets Claude users create and share a cryptographic proof of US citizenship by NFC-scanning the biometric chip in their US passport. It's privacy-preserving, independently audited, and already trusted by major companies at scale. My team is ready to work directly with yours, starting today, to wire this in. Everyone here in the US wants their Fable 5 access back. Let's give it to them. Rene

Dear Dario Amodei & Anthropic team, Claude with Fable 5 was the best software product I have ever used. Losing access was a gut punch - and I know I'm far from alone. I want to help bring it back, fast. At Self we've built the missing piece to make Anthropic compliant with the government's directive: a way to gate Fable 5 and Mythos 5 to verified US citizens. lets Claude users create and share a cryptographic proof of US citizenship by NFC-scanning the biometric chip in their US passport. It's privacy-preserving, independently audited, and already trusted by major companies at scale. My team is ready to work directly with yours, starting today, to wire this in. Everyone here in the US wants their Fable 5 access back. Let's give it to them. Rene

Rene ◘

359,612 просмотров • 1 месяц назад

Head of Anthropic Labs: "Fable is one of those models you'll just remember - like Sonnet 3.5, Opus 4, Opus 4.5. Fable is way smarter than me. It finishes a task, lists its trade-offs, and I have to ask it to explain them to me." in 20-minute talk, the Anthropic head reveals the full capabilities of the Fable 5 model in practice. Watch it now, then read how to use this frontier model in the article below

Head of Anthropic Labs: "Fable is one of those models you'll just remember - like Sonnet 3.5, Opus 4, Opus 4.5. Fable is way smarter than me. It finishes a task, lists its trade-offs, and I have to ask it to explain them to me." in 20-minute talk, the Anthropic head reveals the full capabilities of the Fable 5 model in practice. Watch it now, then read how to use this frontier model in the article below

Movez

33,774 просмотров • 12 дней назад

A few weeks ago the US government forced Claude Fable 5 offline. It's back. And before they switch it off again, this is your shot to point the most powerful AI on earth at something of your own. Take the 26-year-old in Toronto. His bot went dark the day Fable 5 got pulled. The day it came back, it woke up and started printing again - $568,000 profit and counting. His wallet: Here's the story. When the ban hit, Anthropic had to switch Fable 5 off for everyone. They called it a misunderstanding with the government. Whatever it was, every bot running on that model stopped cold. His was one of them. He didn't rebuild a thing. The loop just sat there, paused, for weeks. Here's what that loop actually does. It's not one prompt to an AI. It's an agentic loop: a team of agents running on repeat. One reads the market, one places the trade, one tries to prove it wrong before a cent moves. Then it goes again, around the clock, keeping what works and dropping what doesn't. Fable 5 is the brain running that team. The exact model the government decided was too powerful to leave on. Point it at Bitcoin's 5-minute markets and it barely misses. Most of its trades close green, and it runs itself. He checks it from his phone. The government could switch off the model. It couldn't switch off the idea. Now the model's back, and so is the money. Save this before next headline. You'll want it the day Fable 5 hits offline again.

A few weeks ago the US government forced Claude Fable 5 offline. It's back. And before they switch it off again, this is your shot to point the most powerful AI on earth at something of your own. Take the 26-year-old in Toronto. His bot went dark the day Fable 5 got pulled. The day it came back, it woke up and started printing again - $568,000 profit and counting. His wallet: Here's the story. When the ban hit, Anthropic had to switch Fable 5 off for everyone. They called it a misunderstanding with the government. Whatever it was, every bot running on that model stopped cold. His was one of them. He didn't rebuild a thing. The loop just sat there, paused, for weeks. Here's what that loop actually does. It's not one prompt to an AI. It's an agentic loop: a team of agents running on repeat. One reads the market, one places the trade, one tries to prove it wrong before a cent moves. Then it goes again, around the clock, keeping what works and dropping what doesn't. Fable 5 is the brain running that team. The exact model the government decided was too powerful to leave on. Point it at Bitcoin's 5-minute markets and it barely misses. Most of its trades close green, and it runs itself. He checks it from his phone. The government could switch off the model. It couldn't switch off the idea. Now the model's back, and so is the money. Save this before next headline. You'll want it the day Fable 5 hits offline again.

cvxv666

24,055 просмотров • 14 дней назад

I was scheduled to do a talk on the best way to use Fable 5 today. Here's how that went: > I got rugpulled by the US government (0:00) > what I'm using Fable for (1:38) > looping, and why few can afford it (4:14) > my affordable looping strategy (6:30) > LLM canaries (17:29)

I was scheduled to do a talk on the best way to use Fable 5 today. Here's how that went: > I got rugpulled by the US government (0:00) > what I'm using Fable for (1:38) > looping, and why few can afford it (4:14) > my affordable looping strategy (6:30) > LLM canaries (17:29)

Rob Hallam

32,211 просмотров • 1 месяц назад

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

151,786 просмотров • 1 месяц назад

CLAUDE FABLE 5 IS BACK, BUT IS IT THE SAME POWER OR WAS IT QUIETLY TRIMMED Fable 5 is available again. Anthropic's most powerful model is back globally for every paid account When it first landed three weeks ago, the difference was obvious right away. It didn't just help, it carried the task to the finish: entire web projects, presentations, reports, analysis. People dropped their work on other models and simply waited for Fable And the wait happened because three days after launch the US hit the model with export controls. There was no clean way to cut off foreign nationals, so Anthropic shut access down for everyone Now the restrictions are lifted, and here are the terms it came back on: > until July 7 it's included in a plan and covers up to 50% of the weekly limit > after that only through credits, topped up in settings > output runs at 50 dollars per million tokens Anthropic says it's the same model, just with stronger protection against cybersecurity use. Because the filter is so strict, there will be false positives now and then, and a request will get bumped down to a weaker model Whether it's the same power or quietly trimmed will only show after a couple of weeks of real work

CLAUDE FABLE 5 IS BACK, BUT IS IT THE SAME POWER OR WAS IT QUIETLY TRIMMED Fable 5 is available again. Anthropic's most powerful model is back globally for every paid account When it first landed three weeks ago, the difference was obvious right away. It didn't just help, it carried the task to the finish: entire web projects, presentations, reports, analysis. People dropped their work on other models and simply waited for Fable And the wait happened because three days after launch the US hit the model with export controls. There was no clean way to cut off foreign nationals, so Anthropic shut access down for everyone Now the restrictions are lifted, and here are the terms it came back on: > until July 7 it's included in a plan and covers up to 50% of the weekly limit > after that only through credits, topped up in settings > output runs at 50 dollars per million tokens Anthropic says it's the same model, just with stronger protection against cybersecurity use. Because the filter is so strict, there will be false positives now and then, and a request will get bumped down to a weaker model Whether it's the same power or quietly trimmed will only show after a couple of weeks of real work

Mikadzyki🌙

19,370 просмотров • 13 дней назад

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

Chris

228,126 просмотров • 26 дней назад

This is crazy man... one prompt + 4 minutes and I hit my 5 hour rate limit in the Gemini app. Here is the proof. Yesterday, I hit the rate limit the same way, so I wanted to check if it would happen again. I started with 0% usage on my 5 hour limit, then gave one simple prompt for video generation using the avatar feature. It ran for around 3 to 4 minutes, hit 100% of the rate limit, and the video generation failed as well.

AshutoshShrivastava

82,414 просмотров • 1 месяц назад

FOR THE FIRST TIME, THE US GOVERNMENT PULLED A PUBLICLY DEPLOYED AI MODEL OFFLINE. THE MARKET SHOULD PAY ATTENTION. 🇺🇸 On June 12, the Commerce Department ordered Anthropic to block access to its newest models, Fable 5 and Mythos 5, for any foreign national. Anthropic had to disable them for everyone to comply. The stated reason was a security concern. Anthropic disputes it, calling the flagged technique a narrow jailbreak available in other models too. Whatever the truth, here's the macro takeaway: Frontier AI now sits close enough to national security that a government can switch it off overnight. If you don't control your own models and compute, you're one directive away from losing access. That's the case for open and local models, in one headline.

FOR THE FIRST TIME, THE US GOVERNMENT PULLED A PUBLICLY DEPLOYED AI MODEL OFFLINE. THE MARKET SHOULD PAY ATTENTION. 🇺🇸 On June 12, the Commerce Department ordered Anthropic to block access to its newest models, Fable 5 and Mythos 5, for any foreign national. Anthropic had to disable them for everyone to comply. The stated reason was a security concern. Anthropic disputes it, calling the flagged technique a narrow jailbreak available in other models too. Whatever the truth, here's the macro takeaway: Frontier AI now sits close enough to national security that a government can switch it off overnight. If you don't control your own models and compute, you're one directive away from losing access. That's the case for open and local models, in one headline.

Merlijn The Trader

47,986 просмотров • 1 месяц назад

On a previous #TeawithGaryVee I was asked "My YouTube channel hit 485 subscribers in 3 months and then just hit a wall. Any thoughts?" If you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on

On a previous #TeawithGaryVee I was asked "My YouTube channel hit 485 subscribers in 3 months and then just hit a wall. Any thoughts?" If you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on

Gary Vaynerchuk

13,300 просмотров • 8 месяцев назад

Fable 5 is back. So I gave it one of my favorite frontend tests: one prompt + one reference image. The task: rebuild a 3D globe dashboard.🌎 The result was almost too good. It did not just make a working page. The texture, lighting, spacing, depth, glass panels, and overall layout were all close to the reference. Some parts felt nearly pixel-level.

Fable 5 is back. So I gave it one of my favorite frontend tests: one prompt + one reference image. The task: rebuild a 3D globe dashboard.🌎 The result was almost too good. It did not just make a working page. The texture, lighting, spacing, depth, glass panels, and overall layout were all close to the reference. Some parts felt nearly pixel-level.

Kai

113,304 просмотров • 13 дней назад

“So it was a wheel shield problem. You hit the curb?” Kimi: “I hit the curb like every lap. That lap I probably hit it less even than previous laps. And I could feel instantly something broke. And I just didn't know what was going on. And also the team was trying to understand. Initially, we thought it was the front wing, but then it wasn't the front wing. Yeah, just didn't didn't go our way. And it was a shame because I think we had a real shot for the win today. It's one extra motivation to be back stronger and to do even better.”

“So it was a wheel shield problem. You hit the curb?” Kimi: “I hit the curb like every lap. That lap I probably hit it less even than previous laps. And I could feel instantly something broke. And I just didn't know what was going on. And also the team was trying to understand. Initially, we thought it was the front wing, but then it wasn't the front wing. Yeah, just didn't didn't go our way. And it was a shame because I think we had a real shot for the win today. It's one extra motivation to be back stronger and to do even better.”

did kimi score points in f1 today?

290,615 просмотров • 10 дней назад

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

stevibe

398,628 просмотров • 2 месяцев назад

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

Joey

22,855 просмотров • 2 месяцев назад

This is one of the wildest things I have built with AI so far 😱 Asked Claude Fable 5 to create a fully interactive 3D crime investigation board using Three.js and it did it in ONE Prompt. It took one more prompt to give it a polished UI inspired by a reference design that I created with GPT Image. What surprised me most was not how it looked. It was how it worked. Fable built a complete Verlet rope physics system with 46 particles in every thread. It handled gravity, damping, distance constraints and all other features of file upload, text and everything. The threads/ropes even lift naturally with pinned evidence instead of clipping through it. I never asked it to implement any of that. I only described the experience I wanted. Maybe that's how we should work with AI. Instead of writing long long specs, we should explain what we want as an experience and let AI do its thing. Anyway, every day Fable 5 raises the bar for what I think AI can build. More to this Live: Code:

This is one of the wildest things I have built with AI so far 😱 Asked Claude Fable 5 to create a fully interactive 3D crime investigation board using Three.js and it did it in ONE Prompt. It took one more prompt to give it a polished UI inspired by a reference design that I created with GPT Image. What surprised me most was not how it looked. It was how it worked. Fable built a complete Verlet rope physics system with 46 particles in every thread. It handled gravity, damping, distance constraints and all other features of file upload, text and everything. The threads/ropes even lift naturally with pinned evidence instead of clipping through it. I never asked it to implement any of that. I only described the experience I wanted. Maybe that's how we should work with AI. Instead of writing long long specs, we should explain what we want as an experience and let AI do its thing. Anyway, every day Fable 5 raises the bar for what I think AI can build. More to this Live: Code:

The Bugged Dev

92,073 просмотров • 4 дней назад

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

Reese Chong

52,588 просмотров • 2 месяцев назад

dflash-mlx v0.1.7 is out. Big adaptive-runtime update, still focused mostly on Qwen3.6 27B 4-bit. @ 2048 tokens, M5 Max, stock mlx_lm baseline: ► 1024: 33.26 → 98.05 tok/s (x2.95) ► 2048: 32.34 → 90.67 tok/s (x2.81) ► 4096: 30.58 → 93.55 tok/s (x3.06) ► 8192: 26.03 → 79.12 tok/s (x3.04) ► 16384: 21.50 → 60.77 tok/s (x2.78) Main change: adaptive verify got a lot smarter. Instead of blindly trying to verify large 16-token blocks all the time, DFlash now watches acceptance + tokens/cycle + real cycle cost. When the draft gets weaker, it drops to smaller 4-token blocks, then probes back up only when the recent cycles make sense. In practice: less wasted verify work, better long-context behavior, and much more useful metrics to understand what is happening. ► retuned adaptive verify for long-context / agentic decode ► richer metrics: tokens/cycle, adaptive block state, CopySpec counters ► /metrics now has real decode avg + logical/real/restored prefill rates ► AIME25 benchmark suite with exact integer scoring ► Qwen thinking default now follows tokenizer/request behavior ► GDN recurrent exactness fixes I also started running AIME25-style long generations. Even around 45k generated tokens, I was still seeing ~40 tok/s on 27B 4-bit. Over the next few days I’ll share more demos: AIME runs, real OpenCode game/project sessions, and full metrics along the way. Still optimizing hard for 27B 4-bit first, while working on custom kernels per Apple GPU generation so more machines can benefit.

dflash-mlx v0.1.7 is out. Big adaptive-runtime update, still focused mostly on Qwen3.6 27B 4-bit. @ 2048 tokens, M5 Max, stock mlx_lm baseline: ► 1024: 33.26 → 98.05 tok/s (x2.95) ► 2048: 32.34 → 90.67 tok/s (x2.81) ► 4096: 30.58 → 93.55 tok/s (x3.06) ► 8192: 26.03 → 79.12 tok/s (x3.04) ► 16384: 21.50 → 60.77 tok/s (x2.78) Main change: adaptive verify got a lot smarter. Instead of blindly trying to verify large 16-token blocks all the time, DFlash now watches acceptance + tokens/cycle + real cycle cost. When the draft gets weaker, it drops to smaller 4-token blocks, then probes back up only when the recent cycles make sense. In practice: less wasted verify work, better long-context behavior, and much more useful metrics to understand what is happening. ► retuned adaptive verify for long-context / agentic decode ► richer metrics: tokens/cycle, adaptive block state, CopySpec counters ► /metrics now has real decode avg + logical/real/restored prefill rates ► AIME25 benchmark suite with exact integer scoring ► Qwen thinking default now follows tokenizer/request behavior ► GDN recurrent exactness fixes I also started running AIME25-style long generations. Even around 45k generated tokens, I was still seeing ~40 tok/s on 27B 4-bit. Over the next few days I’ll share more demos: AIME runs, real OpenCode game/project sessions, and full metrics along the way. Still optimizing hard for 27B 4-bit first, while working on custom kernels per Apple GPU generation so more machines can benefit.

bstn 👁️

16,334 просмотров • 1 месяц назад

Day 4: OpenClaw 🦞 A month ago I was coding solo. Today I have a 9-agent AI team running 24/7. I was mid-optimization when I hit the wall: - Google AI Studio: 0 tokens. - Antigravity: Banned. At 25, I’m not waiting for an API key to let me build. Spent the afternoon pivoting. I'm now hosting the LLM locally. It works. 🛠️ No more rate limits. No more bans. Just pure execution. The game has officially changed. 🚀

Day 4: OpenClaw 🦞 A month ago I was coding solo. Today I have a 9-agent AI team running 24/7. I was mid-optimization when I hit the wall: - Google AI Studio: 0 tokens. - Antigravity: Banned. At 25, I’m not waiting for an API key to let me build. Spent the afternoon pivoting. I'm now hosting the LLM locally. It works. 🛠️ No more rate limits. No more bans. Just pure execution. The game has officially changed. 🚀

Ziwen

28,030 просмотров • 5 месяцев назад