Loading video...

Video Failed to Load

Go Home

Introduce EAGLE, a new method for fast LLM decoding based on compression: - 3x🚀than vanilla - 2x🚀 than Lookahead (on its benchmark) - 1.6x🚀 than Medusa (on its benchmark) - provably maintains text distribution - trainable (in 1~2 days) and testable on RTX 3090s Playground: Blog: Code: ⚒️First Principle:...

hongyangzh's profile picture

Hongyang Zhang

2,739 subscribers

118,810 views • 2 years ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Jointly announcing EAGLE-3 with SGLang: Setting a new record in LLM inference acceleration! - 5x🚀than vanilla (on HF) - 1.4x🚀than EAGLE-2 (on HF) - A record of ~400 TPS on LLama 3.1 8B with a single H100 (on SGLang) - 1.65x🚀in latency even for large bs=64 (on SGLang) - A new scaling law: more training data, better speedup - Apache 2.0 Paper: Code: SGLang version: ⚒️Takeaway: Introducing training-time test, a novel draft model training technique: we replace feature prediction with direct token prediction and shift from top-layer-only features to multi-layer feature fusion. This approach unlocks a new scaling law previously undiscovered in EAGLE and EAGLE-2. 🙏Acknowledge: We would like to thank the SGLang team (zhyncs Lianmin Zheng Ying Sheng James Liu, Ke Bao, and others LMSYS Org) for their merge and careful evaluation of EAGLE-3 on SGLang. 🤝Want to collaborate? We're a small academic group with limited GPU resources. If you're interested in supporting our next version of EAGLE or would like us to train a preliminary version tailored to a specific model, please get in touch! Joint work with Yuhui Li, Fangyun Wei, and Chao Zhang
0:12

Jointly announcing EAGLE-3 with SGLang: Setting a new record in LLM inference acceleration! - 5x🚀than vanilla (on HF) - 1.4x🚀than EAGLE-2 (on HF) - A record of ~400 TPS on LLama 3.1 8B with a single H100 (on SGLang) - 1.65x🚀in latency even for large bs=64 (on SGLang) - A new scaling law: more training data, better speedup - Apache 2.0 Paper: Code: SGLang version: ⚒️Takeaway: Introducing training-time test, a novel draft model training technique: we replace feature prediction with direct token prediction and shift from top-layer-only features to multi-layer feature fusion. This approach unlocks a new scaling law previously undiscovered in EAGLE and EAGLE-2. 🙏Acknowledge: We would like to thank the SGLang team (zhyncs Lianmin Zheng Ying Sheng James Liu, Ke Bao, and others LMSYS Org) for their merge and careful evaluation of EAGLE-3 on SGLang. 🤝Want to collaborate? We're a small academic group with limited GPU resources. If you're interested in supporting our next version of EAGLE or would like us to train a preliminary version tailored to a specific model, please get in touch! Joint work with Yuhui Li, Fangyun Wei, and Chao Zhang

Hongyang Zhang

41,164 views • 1 year ago

Can $300 RTX3060 be faster than $10,000 A100 for LLM decoding?🧐Yes Introduce EAGLE-2: - 4.26x🚀vanilla (13B) - 1.4x🚀EAGLE (13B) - No extra training than EAGLE - Maintain distribution Paper: Code: Demo:
0:13

Can $300 RTX3060 be faster than $10,000 A100 for LLM decoding?🧐Yes Introduce EAGLE-2: - 4.26x🚀vanilla (13B) - 1.4x🚀EAGLE (13B) - No extra training than EAGLE - Maintain distribution Paper: Code: Demo:

Hongyang Zhang

35,677 views • 1 year ago

EAGLE-3 introduces two key innovations: Training-Time Testing (TTT) and multi-level feature fusion. By removing the feature prediction constraint used in previous EAGLE versions and leveraging semantic features across multiple layers, EAGLE-3 achieves higher acceptance rates, faster generation, and lossless performance. The result? - 5.6× faster than vanilla decoding (13B) Compared to EAGLE-1, EAGLE-3 delivers a 1.8× speedup on the 13B model, with the future EAGLE-4 release expected to further improve decoding efficiency. *Inference on the video conducted on 2x RTX 3090 GPUs at fp16 precision using the Vicuna 13B model.
0:12

EAGLE-3 introduces two key innovations: Training-Time Testing (TTT) and multi-level feature fusion. By removing the feature prediction constraint used in previous EAGLE versions and leveraging semantic features across multiple layers, EAGLE-3 achieves higher acceptance rates, faster generation, and lossless performance. The result? - 5.6× faster than vanilla decoding (13B) Compared to EAGLE-1, EAGLE-3 delivers a 1.8× speedup on the 13B model, with the future EAGLE-4 release expected to further improve decoding efficiency. *Inference on the video conducted on 2x RTX 3090 GPUs at fp16 precision using the Vicuna 13B model.

Eagle

36,689 views • 3 days ago

🧵🚀 New WebAgent Benchmark Alert! 🚀 There is hope for human workers! We released WorkArena++, a new challenging benchmark for WebAgents. Our best agent achieves 0% accuracy on this benchmark, while human evaluators still obtain 94%! 🔗
0:16

🧵🚀 New WebAgent Benchmark Alert! 🚀 There is hope for human workers! We released WorkArena++, a new challenging benchmark for WebAgents. Our best agent achieves 0% accuracy on this benchmark, while human evaluators still obtain 94%! 🔗

Alexandre Lacoste

11,267 views • 1 year ago

🚀 "AI Dreams of the Eyes of Its Creator" by Marc Quinn is now live and available to mint on: This groundbreaking collection from an iconic contemporary artist showcases the first 2 editions in a series of 8, crafted by a cutting-edge AI trained on Quinn’s iconic iris portraits.
0:14

🚀 "AI Dreams of the Eyes of Its Creator" by Marc Quinn is now live and available to mint on: This groundbreaking collection from an iconic contemporary artist showcases the first 2 editions in a series of 8, crafted by a cutting-edge AI trained on Quinn’s iconic iris portraits.

LiveArt

138,393 views • 1 year ago

The Audi F1 team just announced its official name and logo and Revolut is very much in the centre of it. The team principle said: “Today, our project takes on its official identity. The Audi Revolut F1 Team name is a symbol of the combined strength of our teams in Germany, UK and Switzerland, together with our partners. #AudiRevolutF1Team #F1
0:11

The Audi F1 team just announced its official name and logo and Revolut is very much in the centre of it. The team principle said: “Today, our project takes on its official identity. The Audi Revolut F1 Team name is a symbol of the combined strength of our teams in Germany, UK and Switzerland, together with our partners. #AudiRevolutF1Team #F1

Max Karpis

492,643 views • 6 months ago

Running more than one team on Telegram? Create up to 10 separate organizations. Each one its own structure, its own view, its own members. Takes less than a minute and is visible to your whole team the second you set it up. Get the full app for free with beta access. Waitlist in first reply.
0:13

Running more than one team on Telegram? Create up to 10 separate organizations. Each one its own structure, its own view, its own members. Takes less than a minute and is visible to your whole team the second you set it up. Get the full app for free with beta access. Waitlist in first reply.

Amua

90,632 views • 10 days ago

BREAKING: Grok Code Fast 1 takes the #1 spot for the first week of 2026 on the OpenRouter AI model rankings. A strong start to the year 🚀
0:13

BREAKING: Grok Code Fast 1 takes the #1 spot for the first week of 2026 on the OpenRouter AI model rankings. A strong start to the year 🚀

DogeDesigner

17,309 views • 5 months ago

1/ 🧙‍♂️! AIP #67 has passed! 🚀 Introducing the FIRST Liquidity Marketplace powered by MIM technology from 🧙🏼‍♂️, built on HyperEVM Hyperliquid. Welcome PurrSwap!😸 This is a HISTORIC moment for the Abracadabra DAO as it incubates its first AMM on one of the most exciting and fastest growing ecosystems in all of crypto! 🎉 Through this incubation, Abracadabra is positioning itself at the forefront of HyperEVM’s infrastructure and ensuring that MIM and PurrSwap become the liquidity and trading hub backbone of HyperEVM. ⚡ Hyperliquid’s momentum is undeniable, HyperEVM is its exciting new frontier and PurrSwap is the DEX that will solve a critical piece of the puzzle going forward. 🧩 🧙🏼‍♂️+😸=📈
0:10

1/ 🧙‍♂️! AIP #67 has passed! 🚀 Introducing the FIRST Liquidity Marketplace powered by MIM technology from 🧙🏼‍♂️, built on HyperEVM Hyperliquid. Welcome PurrSwap!😸 This is a HISTORIC moment for the Abracadabra DAO as it incubates its first AMM on one of the most exciting and fastest growing ecosystems in all of crypto! 🎉 Through this incubation, Abracadabra is positioning itself at the forefront of HyperEVM’s infrastructure and ensuring that MIM and PurrSwap become the liquidity and trading hub backbone of HyperEVM. ⚡ Hyperliquid’s momentum is undeniable, HyperEVM is its exciting new frontier and PurrSwap is the DEX that will solve a critical piece of the puzzle going forward. 🧩 🧙🏼‍♂️+😸=📈

🧙🏼‍♂️

13,103 views • 1 year ago

We are thrilled to partner with Ligue1UberEats on Coach Ligue 1 - a web3 fantasy football platform developed by Unagi on Chiliz - The Sports Blockchain ⚽️🇫🇷 This launch represents a significant expansion of our fantasy sports offerings, setting a new benchmark for innovation in the genre. With millions of French football fans, this initiative is expanding our games portfolio and it's a crucial step in our mission to onboard web2 users to web3! 🚀
0:16

We are thrilled to partner with Ligue1UberEats on Coach Ligue 1 - a web3 fantasy football platform developed by Unagi on Chiliz - The Sports Blockchain ⚽️🇫🇷 This launch represents a significant expansion of our fantasy sports offerings, setting a new benchmark for innovation in the genre. With millions of French football fans, this initiative is expanding our games portfolio and it's a crucial step in our mission to onboard web2 users to web3! 🚀

Unagi

108,319 views • 2 years ago

🚀 Governments are warming up to blockchain-based technologies like DAOs. Japan is one of the nations leading the pack in setting the stage for DAO adoption by establishing a legal framework – a first among major nations. 🇯🇵 Let's dive into Japan's DAO-related law and explore its potential implications for DAOs and the blockchain industry.
0:20

🚀 Governments are warming up to blockchain-based technologies like DAOs. Japan is one of the nations leading the pack in setting the stage for DAO adoption by establishing a legal framework – a first among major nations. 🇯🇵 Let's dive into Japan's DAO-related law and explore its potential implications for DAOs and the blockchain industry.

EMURGO

25,357 views • 2 years ago

🚀 Top4Fans V2 is now live ! This is just the beginning, with more updates and new features on the way for your best experience as we enter a new era for Top4Fans! 💥
0:16

🚀 Top4Fans V2 is now live ! This is just the beginning, with more updates and new features on the way for your best experience as we enter a new era for Top4Fans! 💥

TOP4FANS

17,414 views • 1 year ago

🚀 Big news! The Soneium 💿 Mainnet is officially live! 🎉 This next-gen blockchain ecosystem, developed by Sony and Startale, merges the power of Ethereum Layer 2 with a focus on scalability, security, and user-friendliness. 🌐 Join us on this exciting journey! 🧵1/10
0:19

🚀 Big news! The Soneium 💿 Mainnet is officially live! 🎉 This next-gen blockchain ecosystem, developed by Sony and Startale, merges the power of Ethereum Layer 2 with a focus on scalability, security, and user-friendliness. 🌐 Join us on this exciting journey! 🧵1/10

UNE💿

34,808 views • 1 year ago

Want 100× on every ORE you farm? Day 5 of the Advent Calendar is the first of it's kind 🎄 Trade in the BTC/ZIG pool on Oroswap and get a 100× ORE multiplier for 20 days. The top 3 wallets by trading volume will win a 100× ORE multiplier, the biggest multiplier on any trading platform, applied to all OREs earned in the next 20 days 🚀🚀 Trade here 👉
0:16

Want 100× on every ORE you farm? Day 5 of the Advent Calendar is the first of it's kind 🎄 Trade in the BTC/ZIG pool on Oroswap and get a 100× ORE multiplier for 20 days. The top 3 wallets by trading volume will win a 100× ORE multiplier, the biggest multiplier on any trading platform, applied to all OREs earned in the next 20 days 🚀🚀 Trade here 👉

Oroswap

19,406 views • 6 months ago

🚀 Big News! 🚀 Congratulations to MCN on its Nasdaq listing! 🎉 With cutting-edge smart recommendation products and innovative trading ecosystem, #MCNGlobal continues to push the boundaries of #CryptoTrading💯 Backed by capital investment, the future looks brighter than ever🎇 #MCNGlobal #CryptoTrading #Nasdaq #Web3
0:13

🚀 Big News! 🚀 Congratulations to MCN on its Nasdaq listing! 🎉 With cutting-edge smart recommendation products and innovative trading ecosystem, #MCNGlobal continues to push the boundaries of #CryptoTrading💯 Backed by capital investment, the future looks brighter than ever🎇 #MCNGlobal #CryptoTrading #Nasdaq #Web3

MCN

14,328 views • 1 year ago

Prediction Market is one of the leading Web3 niches in 2024! With about $4 Billion in trading volume, $192 Million in TVL and millions of users in 2024, prediction protocols have been showing tremendous growth and attracting user adoption in Web3. PolyMarket seems to be leading the pack, with over $175M in TVL, and backing from Vitalik Buterin. However, a majority of Prediction Markets, including PolyMarket, currently lacks the flexibility and capital efficiency required for seamless transactions. Also, they are all majorly focused on driving Web2 users thus neglecting the need for markets that cater for short-term, high-risk investments from Web3 Degens. To tackle these flaws, there's a need for a revolutionary contender that understands the need of Web3 Chads That's where Predict Hub comes in PredictHub is a prediction Market that transforms real world events into opportunities for everyone to participate and forecast. Launching on Arbitrum, PredictHub is already catching the attention of major players in the Space by offering something PolyMarket and others don't - Flexibility and Incentives. By offering fast market updates and innovative prediction category like ETF Forecasts, PredictHub is changing how we interact with Prediction Markets. But then, here's where it gets more interesting; PredictHub offer users a unique point system, where you don't just make predictions, you also earn rewards. The more you Predict, the more you earn. These rewards are 2-fold: Nova and Orbit Points. Nova Points are earned by traders based on their trading activity and their leaderboard ranking. Orbit Points, on the other hand, are earned by users who provide liquidity, based on their LP size and duration. Other Point systems include PolyMarket User Points, Leaderboard Bonus and Market Multipliers. These rewards offer users more competitive edge than other prediction markets. Apart from these rewards, PredictHub features a unique 3-tier referral system, rewarding users with even more as you invite your friends. The more friends you bring, the greater the rewards. On top of these, PredictHub focuses on USDC and a wide-range of yield bearing assets like GLP, gUSDC, and sUSDe, enabling users to optimise their earning while holding assets across Networks. Exciting, right? PredictHub is in its Testnet phase and you can start earning Points Right away 🔅 Here's how to Get Started on PredictHub: 1. Go to 2. Request Faucet 3. Start making predictions and earning Points Easy-Peasy ✅ More Info can be gotten from Predict Hub All eyes are on PredictHub as the fix for the flaws of Prediction Market Protocols. With its unique approach targeting untapped niches that most existing prediction markets have yet to explore, I believe the Protocol has the potential to become a breakout success I will be placing good Predictions to Position 🚀🚀🚀
0:11

Prediction Market is one of the leading Web3 niches in 2024! With about $4 Billion in trading volume, $192 Million in TVL and millions of users in 2024, prediction protocols have been showing tremendous growth and attracting user adoption in Web3. PolyMarket seems to be leading the pack, with over $175M in TVL, and backing from Vitalik Buterin. However, a majority of Prediction Markets, including PolyMarket, currently lacks the flexibility and capital efficiency required for seamless transactions. Also, they are all majorly focused on driving Web2 users thus neglecting the need for markets that cater for short-term, high-risk investments from Web3 Degens. To tackle these flaws, there's a need for a revolutionary contender that understands the need of Web3 Chads That's where Predict Hub comes in PredictHub is a prediction Market that transforms real world events into opportunities for everyone to participate and forecast. Launching on Arbitrum, PredictHub is already catching the attention of major players in the Space by offering something PolyMarket and others don't - Flexibility and Incentives. By offering fast market updates and innovative prediction category like ETF Forecasts, PredictHub is changing how we interact with Prediction Markets. But then, here's where it gets more interesting; PredictHub offer users a unique point system, where you don't just make predictions, you also earn rewards. The more you Predict, the more you earn. These rewards are 2-fold: Nova and Orbit Points. Nova Points are earned by traders based on their trading activity and their leaderboard ranking. Orbit Points, on the other hand, are earned by users who provide liquidity, based on their LP size and duration. Other Point systems include PolyMarket User Points, Leaderboard Bonus and Market Multipliers. These rewards offer users more competitive edge than other prediction markets. Apart from these rewards, PredictHub features a unique 3-tier referral system, rewarding users with even more as you invite your friends. The more friends you bring, the greater the rewards. On top of these, PredictHub focuses on USDC and a wide-range of yield bearing assets like GLP, gUSDC, and sUSDe, enabling users to optimise their earning while holding assets across Networks. Exciting, right? PredictHub is in its Testnet phase and you can start earning Points Right away 🔅 Here's how to Get Started on PredictHub: 1. Go to 2. Request Faucet 3. Start making predictions and earning Points Easy-Peasy ✅ More Info can be gotten from Predict Hub All eyes are on PredictHub as the fix for the flaws of Prediction Market Protocols. With its unique approach targeting untapped niches that most existing prediction markets have yet to explore, I believe the Protocol has the potential to become a breakout success I will be placing good Predictions to Position 🚀🚀🚀

InfoSpace OG

19,655 views • 1 year ago

The EU–Viet Nam Comprehensive Strategic Partnership is more than a title on paper - it delivers concrete benefits for the people of Hanoi. Today, I visited the construction site of a metro station, supported by European funding, with contributions from the European Investment Bank , the European Commission, and France, Germany, and Italy. This is Team Europe in action.
0:19

The EU–Viet Nam Comprehensive Strategic Partnership is more than a title on paper - it delivers concrete benefits for the people of Hanoi. Today, I visited the construction site of a metro station, supported by European funding, with contributions from the European Investment Bank , the European Commission, and France, Germany, and Italy. This is Team Europe in action.

António Costa

18,901 views • 4 months ago

Why on earth does the US keep a for-profit healthcare system that delivers worse outcomes at higher costs than countries with universal healthcare? The answer is infuriating: For-profit healthcare has generated 51 billionaires on Forbes' annual billionaires list, and in a country ruled by profit, creating billionaires is more important than investing in the health and well-being of its own people.
0:14

Why on earth does the US keep a for-profit healthcare system that delivers worse outcomes at higher costs than countries with universal healthcare? The answer is infuriating: For-profit healthcare has generated 51 billionaires on Forbes' annual billionaires list, and in a country ruled by profit, creating billionaires is more important than investing in the health and well-being of its own people.

Power to the People ☭🕊

59,808 views • 20 days ago

WTF is this? A startup announcing their benchmark in the sky! NEO ( Neo AI ) is shaking up the game and has stayed at #1 with 34.2% for 4 weeks. A multi-agent system with its own reasoning engine that not just plans and codes but executes the entire workflow, reducing the grunt work by 90% and saving ML teams thousands of hours.
0:10

WTF is this? A startup announcing their benchmark in the sky! NEO ( Neo AI ) is shaking up the game and has stayed at #1 with 34.2% for 4 weeks. A multi-agent system with its own reasoning engine that not just plans and codes but executes the entire workflow, reducing the grunt work by 90% and saving ML teams thousands of hours.

AGAFE

57,898 views • 9 months ago