EAGLE-4 is here. From EAGLE-1 to EAGLE-3, we’ve kept making inference faster. EAGLE-4 is our fastest yet. 961.6 tokens/s on Qwen’s Qwen3.8-27B with TokenSpeed (BF16, 4× NVIDIA B300, TP4, one request, T=0). Github:

One request on @Alibaba_Qwen’s Qwen3.8-27B: 4× vanilla throughput. EAGLE-4: 961.6 tokens/s DFlash 2: 672.6 DSpark: 618.6 Vanilla: 240.5 SPEED-Bench qualitative, 4× @NVIDIA B300, TP4, concurrency 1, T=0. MegaKernel disabled, TokenSpeed.

The same trained Qwen3.8-27B drafter, now at 512 concurrent requests: EAGLE-4: 28,340 aggregate tokens/s DFlash 2: 16,641 DSpark: 14,229 Vanilla: 21,203 1.34× vanilla, 1.70× DFlash 2 and 1.99× DSpark. Paper SPEED-Bench comparison, T=0, SGLang.

On @NVIDIA’s Nemotron 3.5 Super VL: Single request: 647 vs 236 tokens/s, 2.74× vanilla. At 512 concurrent requests: 33,480 vs 19,823 aggregate tokens/s, 1.69× vanilla. SPEED-Bench qualitative, 4× B300, TP4, T=0.

EAGLE-4 on @AMD Instinct MI350X. 262.89 vs 75.27 tokens/s for a single request: 3.49× vanilla. At 32 concurrent requests: 2,360.83 vs 1,390.52 aggregate tokens/s, 1.70× vanilla. AMD team’s day-0 results. 1 GPU, TP1; 117-turn SPEED-Bench; max 4,096 new tokens.

Then we put it on a phone. Motorola Razr 70 Ultra CPU: HumanEval: 82.6 vs 22.8 tokens/s, 3.62× vanilla. Long-form generation: 74.7 vs 23.3, 3.21×. 4-bit Gemma-4-E2B-it, single request, T=0, cool device. Unmodified, non-rooted phone.

On that phone’s long-form test, faster generation also used less energy per token. 166 vs 356 mJ/token measured at the USB input: 53.4% less energy per token. Generation finished in 14s vs 43s with vanilla, before thermal throttling began.

EAGLE-Harness profiles drafting and verification costs during warm-up, then uses those measurements and draft confidence to adapt tree depth, width and verification budget at runtime. One drafter per target model, adapting across hardware and batch sizes.

EAGLE Efficient LM Head pairs low-rank vocabulary retrieval with exact scoring of shortlisted candidates. In the profiled setup, draft-forward latency falls from 574 μs to 255 μs: a 55.6% reduction.

Thank you to our partners @NVIDIA, @AMD, @vllm_project , @sgl_project, @lightseekorg, @fal, @FireworksAI_HQ, @gmi_cloud and others. And to the researchers, engineers and everyone else who helped bring EAGLE-4 to developers.

Everything you need to get started with EAGLE-4. Blog: GitHub: Paper: Website: Integrations: vLLM: SGLang: TokenSpeed: Model weights on Hugging Face: Qwen3.8-27B - Nemotron 3.5 Super VL -

@Alibaba_Qwen @nvidia Congratulations on the launch!

@Alibaba_Qwen @nvidia And draft models on HuggingFace are all you need for testing!

@Alibaba_Qwen @nvidia speed matters more than people think

@Alibaba_Qwen @nvidia sheesh

@Alibaba_Qwen @nvidia thats huge guys, congrats with the launch

@Alibaba_Qwen @nvidia This is definitely one of the fastest inferences I’ve seen. Good job!

@Alibaba_Qwen @nvidia

@Alibaba_Qwen @nvidia La única saga en la que cada secuela promete ir más rápido que la anterior.

@Alibaba_Qwen @nvidia 4x on a single request is wild but I’m more interested in the 1.3-1.7x at 512 concurrent. That’s the one that actually matters.

@Alibaba_Qwen @nvidia "Eagle 4 Source-Available License" for an autoregressive drafter with tree methods that have been around since forever? and you trademarked the one thing that may have been useful in the future in new and current speculators? (efficient lm head) shameless

@Alibaba_Qwen @nvidia I’m actually really hopeful that after AI subsidies run out costs will be reasonable based on efficiencies like this. gg

@Alibaba_Qwen @nvidia The more I work with AI recently the more I feel speed is becoming the primary bottleneck. Thank you @EagleCorp

@Alibaba_Qwen @nvidia This is freaking amazing 🔥

@Alibaba_Qwen @nvidia thanks for the github link

@Alibaba_Qwen @nvidia DFlash 2: we scale under load. EAGLE-4: hold my benchmark. 1.70× the throughput at 512 concurrent requests? Okay, now we need to see what developers build with this

@Alibaba_Qwen @nvidia So it's a super fast spec decoding draft model? I'm going to check out your repo and see if I could test it on 27b. Thanks for posting!

@Alibaba_Qwen @nvidia open source inference keeps getting better. great work by the team!

@Alibaba_Qwen @nvidia 961.6 tokens/s on one request is a big number Curious how it holds up when more requests come in at once

@Alibaba_Qwen @nvidia Damn this is amazing

@Alibaba_Qwen @nvidia Excited to be one of the official launch partners!

@Alibaba_Qwen @nvidia The waiting cursor has enemies now.

@Alibaba_Qwen @nvidia Speed is everything for me, love this guys!

@Alibaba_Qwen @nvidia The engineering behind adaptive tree verification is just as interesting as the speed numbers.

@Alibaba_Qwen @nvidia 961 tokens per second is wild. Inference speed is moving fast.

@Alibaba_Qwen @nvidia These results could have a meaningful impact on the cost of serving AI models.

@Alibaba_Qwen @nvidia nearly 4x vanilla throughput on a single request is insane. faster inference could make a real difference for agents that need to generate long responses.

@Alibaba_Qwen @nvidia Faster generation with lower energy usage is exactly the combination worth chasing.

@Alibaba_Qwen @nvidia The long-context section caught my eye.

@Alibaba_Qwen @nvidia Those mobile results are genuinely surprising. Local AI is moving faster than expected.

@Alibaba_Qwen @nvidia These benchmarks highlight how much optimization can improve the overall AI experience.

@Alibaba_Qwen @nvidia holy

@Alibaba_Qwen @nvidia EAGLE-4 is turning inference optimization into something developers can really feel.

@Alibaba_Qwen @nvidia These numbers are seriously impressive. Nearly 4x faster inference, and they're seeing major improvements across GPUs and phones too.

@Alibaba_Qwen @nvidia More benchmarks on ordinary hardware, please.