Jointly announcing EAGLE-3 with SGLang: Setting a new record... in LLM inference acceleration! - 5x🚀than vanilla (on HF) - 1.4x🚀than EAGLE-2 (on HF) - A record of ~400 TPS on LLama 3.1 8B with a single H100 (on SGLang) - 1.65x🚀in latency even for large bs=64 (on SGLang) - A new scaling law: more training data, better speedup - Apache 2.0 Paper: Code: SGLang version: ⚒️Takeaway: Introducing training-time test, a novel draft model training technique: we replace feature prediction with direct token prediction and shift from top-layer-only features to multi-layer feature fusion. This approach unlocks a new scaling law previously undiscovered in EAGLE and EAGLE-2. 🙏Acknowledge: We would like to thank the SGLang team (zhyncs Lianmin Zheng Ying Sheng James Liu, Ke Bao, and others LMSYS Org) for their merge and careful evaluation of EAGLE-3 on SGLang. 🤝Want to collaborate? We're a small academic group with limited GPU resources. If you're interested in supporting our next version of EAGLE or would like us to train a preliminary version tailored to a specific model, please get in touch! Joint work with Yuhui Li, Fangyun Wei, and Chao Zhangshow more

Introduce EAGLE, a new method for fast LLM decoding... based on compression: - 3x🚀than vanilla - 2x🚀 than Lookahead (on its benchmark) - 1.6x🚀 than Medusa (on its benchmark) - provably maintains text distribution - trainable (in 1~2 days) and testable on RTX 3090s Playground: Blog: Code: ⚒️First Principle: Compression! Yi Ma We find that the sequence of second-top-layer features is compressible, making the prediction of subsequent feature vectors from previous ones easy by a small model. 🙏Acknowledge: This project is greatly inspired by the Medusa team (Tianle Cai @yli3521 Zhengyang Geng Hongwu Peng Tri Dao), the Lookahead team (Hao Zhang LMSYS Org), and others. Joint work with Yuhui Li and Chao Zhangshow more

🚀 Self-speculation brings 6.75x real speedup for LLM generation... with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:show more

Can $300 RTX3060 be faster than $10,000 A100 for... LLM decoding?🧐Yes Introduce EAGLE-2: - 4.26x🚀vanilla (13B) - 1.4x🚀EAGLE (13B) - No extra training than EAGLE - Maintain distribution Paper: Code: Demo:show more

Added a fun lil widget to the LLM Engineer's... Almanac -- a "Token Timing Simulator" so you can get a visceral feel for what a benchmark perf number means. Here's David Wang's latest work with Zhijian Liu's DFlash technique in SGLang -- ~1k TPS!show more

Grok 2 mini is now 2x faster than it... was yesterday. In the last three days Lianmin Zheng and Saeed Maleki rewrote our inference stack from scratch using SGLang ( This has also allowed us to serve the big Grok 2 model, which requires multi-host inference, at a reasonable speed. Both models didn’t just get faster, but also slightly more accurate. Stay tuned for further speed improvements!show more

Martial eagle (Slide 1) is stronger than bald eagle... (Slide 2). It is also a better hunter. It can take down larger, more challenging prey like antelopes or monitor lizards, unlike the bald eagle which relies more on fish and scavenging. The martial eagle’s predatory style is also more aggressive and versatile. Why is the bald eagle now used more by people? That is the power of marketing. The bald eagle is the national bird of the United States. It has been packaged in movies and clips to look more regal and powerful. They even replace its sound with a louder and more majestic call of a red tailed hawk. Watch the second clip and you'll hear their natural sounds. Also, when you type "eagle," the emoji that comes out is 🦅 which is that of bald eagle. That's the power of PR.show more

1/6 Saving Eddie The Eagle - A Thread Eddie... the Eagle was in terrible pain & all alone with what looked like a broken leg The ambulance arrived to help him Read on 🧵⬇️show more

Chris Gotterup moves up to 4th with a long... putt for eagle on 12.show more

(1/n) 🚀 With FastVideo, you can now generate a... 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licensesshow more

It’s a great time to be an eagle! Extremely... excited about our guys returning(0 portal entries) and the class we have coming in. Still looking to add 3 impact transfer arms. DM if are looking for a quality institution in the south with top notch facilities/ technology!show more

A bald eagle was caught on video flying off... with a squirrel near 122nd and Throop in Chicago’s West Pullman neighborhood. Credit CHICAGO CRITTERshow more

American Eagle CRJ-900 had to be evacuated at Augusta... Regional Airport on Tuesday morning after smoke entered the cabin while taxiing to the gate. “American Eagle flight 5406, operated by PSA Airlines, with service from Charlotte Douglas International Airport (CLT) to Augusta Regional Airport (AGS), experienced a maintenance issue after landing safely in Augusta. All passengers deplaned and were escorted to the airport terminal. The safety of our customers is our top priority, and we apologize for the inconvenience,” a spokesperson for American Airlines said in a statement.show more

Plyometric Training for Hoopers 🏀🚀🚀 Basketball involves a lot... of change of direction and is played in multiple planes. The goal is to create better and more efficient movers on the court. Here are a couple variations the will help with change of direction and explosivenessshow more

Another chip in eagle for Minjee?!?! This time on... a par 4?!?! 🤯 🤯 🤯show more

It’s a total free for all in Eagle Pass... right now. Mass illegal crossing taking place for over an hour and a half. Almost 2 years to the day we saw 15,000+ Haitians under the bridge in Del Rio, we now have thousands of predominantly Venezuelans gathering under Eagle Pass bridge.show more

EAGLE SOARS: New Jersey now has more than 250... bald eagle pairs in all 21 counties, up from just one nest in the 1970s, according to the state Department of Environmental Protection and Fish & Wildlife. READ MORE: #News12NJ #NewJerseyNewsshow more

Call of the martial eagle (Polemaetus bellicosus) a large... eagle native to sub-Saharan Africa, It is a powerful hunter that hunts from high soar in a steep stoop. 📹 Jiri Zayferus / zayferusshow more

The ecstasy and pain of golf right here. Ronni... Yin, in a 5 woman playoff, reached the par 5 18th in two with this insane swing. You couldn’t get the ball closer. That putt for Eagle and a major victory. She would then 3 putt and lose. 😢show more

Tbh.. I didn’t know these existed. A couple of... these are pretty close design wise with what I had in mind.. pretty much armadillo and eagle are dead on with a slight color difference and I was planning to have x have more armor on the back.. not sure if I should continue now.show more

World modeling and imitation learning have largely been considered... two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -show more