EAGLE-3 introduces two key innovations: Training-Time Testing (TTT) and... multi-level feature fusion. By removing the feature prediction constraint used in previous EAGLE versions and leveraging semantic features across multiple layers, EAGLE-3 achieves higher acceptance rates, faster generation, and lossless performance. The result? - 5.6× faster than vanilla decoding (13B) Compared to EAGLE-1, EAGLE-3 delivers a 1.8× speedup on the 13B model, with the future EAGLE-4 release expected to further improve decoding efficiency. *Inference on the video conducted on 2x RTX 3090 GPUs at fp16 precision using the Vicuna 13B model.show more

Introduce EAGLE, a new method for fast LLM decoding... based on compression: - 3x🚀than vanilla - 2x🚀 than Lookahead (on its benchmark) - 1.6x🚀 than Medusa (on its benchmark) - provably maintains text distribution - trainable (in 1~2 days) and testable on RTX 3090s Playground: Blog: Code: ⚒️First Principle: Compression! Yi Ma We find that the sequence of second-top-layer features is compressible, making the prediction of subsequent feature vectors from previous ones easy by a small model. 🙏Acknowledge: This project is greatly inspired by the Medusa team (Tianle Cai @yli3521 Zhengyang Geng Hongwu Peng Tri Dao), the Lookahead team (Hao Zhang LMSYS Org), and others. Joint work with Yuhui Li and Chao Zhangshow more

New blog! 🚀 MTP, EAGLE-3, DFlash or DSpark, which... speculative decoding method should you actually use? There’s no universal winner. The best choice changes with the model, workload, and speculation depth. We break down how 5 methods work, how to enable and tune them in vLLM, and benchmark them across Gemma, Qwen, Kimi and MiniMax on AMD Instinct MI300X & MI355X. Deep dive 👇show more

🔥 Final Hours of the Event with the Highest... Staking Rates in Sl8’s History Sl8’s special event ends today, August 4, at 23:59 UTC. Until then, you can still activate staking and lock in the increased rate for the entire selected term: • 72.5% annually — 90 days • 97.5% annually — 180 days • 120% annually — 360 days Rewards are paid weekly. Once the event ends, these terms will no longer be available. Now is the time to check your position in the TOP-100 The final leaderboard will determine which prize each participant receives. The prize pool includes tokenized gold, platinum, and silver coins. ● 1st place 3 × 1 oz American Gold Eagle Total value: approximately $12,212 ● 2nd place 2 × 1 oz American Gold Eagle Total value: approximately $8,808 ● 3rd–9th places 1 × 1 oz American Gold Eagle Value: approximately $4,404 ● Every 10th place: 10, 20, 30…100 1 × Platinum Eagle Value: approximately $1,919 ● 15th, 25th, 35th, 45th, 55th, 65th, 75th, 85th, and 95th places 1 × 1/10 oz American Gold Eagle Value: approximately $440 ● All other TOP-100 participants 1 × American Silver Eagle Value: approximately $68 Sign in on the event page to check your current position and, if necessary, improve your ranking before the leaderboard closes. ● Staking rates, leaderboard, prizes, and full terms: Only a few hours remain. Activate staking before 23:59 UTC to lock in up to 120% annually for the entire selected term.show more

🚀 Self-speculation brings 6.75x real speedup for LLM generation... with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:show more

Over 1000 people are racing from New York City... to Boston tomorrow. And nobody is talking about it. The Five Sisters Cup will feature 4/5 tall ships designed to a common specification by Blohm & Voss in Hamburg, Germany. Each barque has 3 masts, and a crew over 250. Gorch Fock II 🇩🇪 Eagle 🇺🇸 Sagres 🇵🇹 Mircea 🇷🇴 The Gorch Fock I, Eagle, Sagres, and Mircea are 1930s contemporaries. The first three were purpose-built training vessels for Germany with Mircea built for Romania. After the war, the German vessels were redistributed as reparations. Gorch Fock II was built as a replacement for Gorch Fock I in the 1950s. Unfortunately, Gorch Fock I is no longer seaworthy and currently resides in Stralsund as a museum. The last time all these ships saw one another was for the 🇺🇸 Bicentennial in 1976. Gorch Fock II claimed the Five Sisters Cup then, but who will claim it in the days ahead?show more

Batch Normalization by hand ✍️ ~ 7 steps walkthrough... below Batch normalization is common practice for improving training and achieving faster convergence. It sounds simple. But it is often misunderstood. 🤔 Does batch normalization involve trainable parameters, tunable hyper-parameters, or both? 🤔 Is batch normalization applied to inputs, features, weights, biases, or outputs? 🤔 How is batch normalization different from layer normalization? So I drew and calculated one entirely by hand. Goal: normalize a mini-batch of 4 examples to mean 0 and variance 1, then let the network scale it back. = 1. Given = A mini-batch of 4 training examples, each with 3 features. = 2. Linear layer = Let us multiply by the weights and add the biases. Batch norm sits after this, which answers the second question: what gets normalized is features, not inputs, weights or biases. = 3. ReLU = We apply the activation, and -2 becomes 0. Negative values are suppressed before any statistic is taken. = 4. Batch statistics = Let us compute the sum, mean, variance and standard deviation, one row at a time. A row is a feature and the four columns are the four examples, so every number here measures one feature against the rest of the batch. That is the "batch" in batch normalization, and it is exactly what layer normalization does not do. The statistics are rounded to whole numbers, which is what keeps the rest of the page doable in pen. = 5. Shift to mean 0 = We subtract the mean, in green. The four values in each feature now average to zero. = 6. Scale to variance 1 = Let us divide by the standard deviation, in orange. Each feature now has variance one, whatever scale it arrived at. = 7. Scale and shift = We multiply by a linear transformation and pass the result on. The diagonal and the last column are trainable, so having just forced every feature to mean 0 and variance 1, we hand the network the means to undo it. The outputs: Mean of each feature = [2, 1, 2] Std dev of each feature = [1, 1, 2] To the next layer = [2, -2, 2, 0], [-3, 3, 6, -3], [2, 0, 1, 2] The answers: 🤔 Both. The scale and shift are trainable, the statistics are not. Epsilon and the momentum on the running statistics are the hyper-parameters, and one mini-batch by hand needs neither. 🤔 Features, after the linear layer, not inputs, weights or biases. 🤔 Batch norm measures across the batch, one feature at a time. Layer norm measures across the features, one example at a time. 💾 Save this post!show more

My dual RTX PRO 6000 setup is currently training... a Draft model for Qwen 3.6 27B! 🔥 I'm taking the paper DeepSeek dropped on 6/26 and going for a super ambitious application to the 27B scale. Thanks to my homelab, I was able to dive straight in — I read the paper and immediately started experimenting. The amount I've learned has been insane: - How memory bandwidth bottlenecks speed and clever ways to hack around it - Methods to train the draft model and boost its accuracy - Mechanisms to reference tokens all the way back to the previous one to skyrocket draft acceptance rates - The impact of Attention vs. GateDeltaNet on speculative decoding performance and how to handle those differences - The unique approaches and trade-offs of MTP, Dflash, JetSpec, and DSpark I could go on forever, but just from speculative decoding alone I've learned so much. The 27B architecture feels way more DSpark-native than JetSpec, so once draft training finishes, I'm going all-in with DSpark! My goal is to beat existing speculative decoding speeds outright — no task-specific shortcuts or cheating, pure general improvement. If you're into this kind of research, I'd love to hear your thoughts, impressions, and any suggestions — please reply! 🚀show more

we sped up distributed inference by up to 5x... with decentralized speculative decoding. many don't realize that AI models normally generate text one single word at a time, waiting for the network after every word. speculative decoding changes this by using a "guess & confirm" system, similar to autocomplete. how it's done: 1. draft locally (the guess) instead of waiting for the network, a tiny, fast model on your device guesses the next few words instantly, without waiting for the network. 2. confirm remotely (the check) the massive remote model doesn't generate from scratch; it just checks the draft. it looks at the guesses in a batch and says "yes, yes, no." you get multiple words in the time it usually takes to get one. 3. adaptive logic dsd is smart. if the topic is creative, it lets the draft flow loose. if the topic is math or code, it checks more strictly. it balances speed and precision automatically so your inference almost feel instant. find out more: paper: blog:show more

🚀 A better, faster co-folding-based binding affinity model. Predicting... how tightly a drug candidate binds to its target is critical in drug discovery. It also requires massive computational resources. State-of-the-art models can take 20 seconds to a minute per prediction, impractical for the demands of large scale early-stage programs . 💠 Today, Recursion’s Valence Labs is releasing Nesso-1: the fastest open-source co-folding-based binding affinity model available. At 1 second per prediction, it’s roughly 20x faster than our previous collaboration on Boltz-2 while matching or surpassing its accuracy across public and internal benchmarks. By leveraging NVIDIA Healthcare cuEquivariance, we’ve been able to further accelerate both training and inference by an additional 2-3x. We look forward to continuing to improve Nesso-1 in collaboration with NVIDIA. Weights and code are fully open-sourced. The core architectural ideas behind Nesso-1 build on the insight that coarse-grained co-folding representations can match full-atom models for affinity prediction at a fraction of the cost. Nesso-1 is the first open implementation of this approach with no proprietary dependencies, trained entirely on public data, built to be reproducible and extensible. We’re already using Nesso-1 internally in active drug discovery programs. Fast, reliable affinity prediction at scale is foundational to the kind of autonomous design loops that define our vision for Autonomous Precision Design and Nesso-1 is a meaningful step toward that. 👉 Report: 👉 Github: 👉 HF:show more

🚀 Sol-Attn is here! We present a training-free sparse... attention method that accelerates video generation while better preserving quality. Sol-Attn unifies dynamic routing, sparse computation, and approximate correction in a single online-softmax pass: • On-the-fly block thresholding for dynamic yet controllable budgets • Proxy-score reuse to approximate unselected blocks Results (vs dense FlashAttention-3): • Wan 2.1-14B: 2.02× end-to-end • HunyuanVideo-13B: 2.12× end-to-end • LTX 2.3: up to 2.4× end-to-end When integrated into Sol-Engine (with kernel fusion + caching): • Wan 2.1-14B: 3.48× end-to-end • HunyuanVideo-13B: 5.08× end-to-end Already available in Sol-Engine. The B200 kernel is still under further optimization. 🎬 Project: 📄 Paper: 🔗 Code:show more

Congratulations to Telangana and the entire EAGLE Force, ably... led by Sandeep Shandilya, for being awarded the Best ANTF(Anti-Narcotics Task Force) in the country by the Hon’ble Union Home Minister Shri Amit Shah at the 3rd National Conference of ANTF Heads in New Delhi. 💐💐 The two-day conference is deliberating on the effective implementation of the Vision Document on Narcotics Control (2026–29), achieving the targets envisaged in it and strengthening our collective resolve towards a Drug-Free #India. Tackling Synthetic drugs and clan labs, Dismantling drug cartels, and strengthening darknet and crypto investigations are among the key areas on the agenda. With this, #Telangana Police has reached first place in our two top priority areas - cyber crime performance and the anti-narcotics performance. We are now trying to improve our ranking in other areas where we have fallen behind like implementation of national criminal laws / ICJS, CCTNS through a special drive and focus. We ll be there soon ! A proud achievement for the entire force. Keep it up, Team TG Police. 👍🏽👏show more

The telephoto system on the vivo X300 Ultra is... a bit insane. This generation brings upgrades not only in sensor specifications, but also in stabilization and subject tracking, pushing smartphone telephoto photography closer to the level of professional cameras. 1. Upgraded 200MP “Thanos Telephoto” system The X300 Ultra uses a Blueprint × Samsung HP0 200-megapixel telephoto sensor, representing the fifth generation of the Zeiss-branded “Thanos telephoto.” Building on the previous HPE solution, vivo further refined the sensor through deep customization, improving color rendering, autofocus, HDR performance, and power efficiency, allowing this 200MP telephoto to deliver stronger overall imaging performance. 2. Gimbal-level stabilization and 60fps tracking capture The X300 Ultra introduces around 3° optical stabilization on its telephoto lens, a massive jump compared with the 0.7°–1° typically seen in telephoto OIS systems and even beyond the 1.2° on the X200 Ultra. This enables CIPA 7.0-level professional stabilization. The stronger stabilization significantly improves handheld shooting at long focal lengths, including 200mm, 400mm, and beyond. In addition, the device features a Blueprint high-refresh tracking engine that supports 60fps motion capture, doubling the industry’s common 30fps capability and allowing telephoto cameras not only to capture static subjects, but also to reliably track fast-moving targets.show more

🚨NEWS: $TSLA DEVELOPING NEW SMALLER, CHEAPER EV SUV •... Tesla is developing an all-new compact electric SUV, according to four people familiar with the matter. • The vehicle is not a variant of the Model 3 or Model Y. ✅ Key Details • Length: approximately 4.28 meters (14 feet) — significantly shorter than the Model Y (about 15.7 feet). • Planned production locations: primarily China (Shanghai factory), with potential expansion to the United States and Europe. • Expected pricing: substantially lower than the entry-level Model 3 (which starts at ~$37,000 in the US and $34,000 in China). ✅ Design Approach • Smaller battery for cost reduction (resulting in shorter range than the Model Y’s 306–327 miles). • Single electric motor instead of dual motors. • Significantly lighter weight target (~1.5 metric tons vs. ~2 tons for Model Y). ✅ Strategic Context • This project comes after Elon Musk scrapped a previous low-cost EV plan in 2024 to focus on robotaxis and Optimus. • The new SUV could be offered in both fully autonomous and human-driven versions to suit different global markets and regulations. • The development is still in early stages, with production unlikely to start in 2026.show more

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming,... live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fungshow more

Microsoft made 100B parameter models run on a single... CPU. bitnet.cpp: The official inference framework for 1-bit LLMs. The math behind 1-bit LLMs is what makes them revolutionary. Traditional LLMs use 16-bit floating point weights. Every parameter is a number like 0.0023847 or -1.4729. When you run inference, you multiply these floats together. Billions of times. That's why you need GPUs, they're optimized for floating point matrix multiplication. BitNet b1.58 uses ternary weights: {-1, 0, 1}. That's not a simplification. That's a fundamental change in the math. When your weights are only -1, 0, or 1: → Multiply by 1 = keep the value → Multiply by -1 = flip the sign → Multiply by 0 = skip entirely Matrix multiplication becomes addition and subtraction. No floating point operations. No GPU required. This is why bitnet.cpp achieves: → 2.37x to 6.17x speedup on x86 CPUs → 1.37x to 5.07x speedup on ARM CPUs → 71.9% to 82.2% energy reduction on x86 → 55.4% to 70.0% energy reduction on ARM The speedups scale with model size. Larger models see bigger gains because there are more operations to simplify. A 100B parameter model running at human reading speed (5-7 tokens/second) on a single CPU. That's not optimization. That's a different paradigm. Why 1.58 bits? Because log₂(3) ≈ 1.58. Three possible values = 1.58 bits of information per weight. The key insight: These models aren't quantized after training. They're trained from scratch with ternary weights. The model learns to work within the constraint. No precision loss. No quality tradeoff.show more

What if you kept asking an LLM to "make... it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and *self-divergence.* In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.show more

Transformer and Mixture of Experts, explained visually! Mixture of... Experts (MoE) is a popular architecture that uses different experts to improve Transformer models. Transformer and MoE differ in the decoder block: - Transformer uses a feed-forward network. - MoE uses experts, which are feed-forward networks but smaller compared to those Transformer. During inference, a subset of experts are selected. This makes inference faster in MoE. Also, since the network has multiple decoder layers: - The text passes through different experts across layers. - The chosen experts also differ between tokens. But how does the model decide which experts should be ideal? The router does that. It is a multi-class classifier that produces softmax scores over experts to select the top K experts. The router is trained with the network, and it learns to select the best experts. But it isn't straightforward. There are challenges! Challenge 1) Notice this pattern at the start of training: - Say, the model selects "Expert 2" - This expert gets a bit better - It may get selected again since it's the "best" - It learns more - It gets selected again in the next iteration - It learns more, and so on! This means many experts can go under-trained due to the overselection of a few experts! We solve this in two steps: - Add noise to the feed-forward output of the router so that other experts can get higher logits. - Set all but the top K logits to -infinity. After softmax, these scores become zero. This way, other experts also get the opportunity to train. Challenge 2) Some experts may get exposed to more tokens than others, leading to under-trained experts. We prevent this by limiting the number of tokens an expert can process. If an expert reaches the limit, the token is passed to the next best expert. Overall, MoEs have more parameters to load. But a fraction of them are activated during inference. This leads to faster inference. Mixtral 8x7B and Llama 4 are two popular MoE-based LLMs. Have you used MoEs in production yet? To dive deeper into how MoE inference works in production, we wrote a full article covering token dispatch, grouped expert computation, model-weight memory, multi-GPU communication, expert placement, load imbalance, and performance diagnosis. Read it below.show more

🧐🇺🇸 The Country That Put More Fighter Jets in... Foreign Skies Than the Rest of the World Combined For more than 70 years, the United States has dominated the global fighter aircraft market, exporting thousands of combat jets to allies across the world. Its fighters have flown in the colors of dozens of nations, making American aircraft the backbone of many of the world's most powerful air forces. The numbers are staggering: ◽ F-16 Fighting Falcon, More than 4,600 built, making it one of the most successful fighter jet programs ever created. It remains America's most exported fighter and is operated by over 25 countries. ◽ F-15 Eagle & Strike Eagle, Among the most capable air superiority and strike fighters ever built, exported to key allies across the Middle East and Asia. ◽ F/A-18 Hornet & Super Hornet, Trusted by allied air forces and navies for decades. ◽ F-35 Lightning II, The largest fifth generation fighter export program in history, with more than 20 nations already committed to the aircraft. America's dominance is not just about building aircraft. It offers pilot training, weapons integration, maintenance support, continuous upgrades, and access to a vast military industrial network that keeps aircraft effective for decades. While many nations can build fighter jets, very few can provide the global support system that comes with them. From the F-4 Phantom and F-5 Tiger during the Cold War to today's F-35 stealth fighter, American aircraft have shaped air combat across generations. No nation has exported more fighter jets. No nation has equipped more allied air forces. And no fighter export empire has had a greater impact on the balance of air power worldwide.show more

Our aero kit for the 2024+ Model 3 Performance... is designed using CFD to enhance performance while preserving Tesla’s renowned efficiency. Shown here is the exploded view of the aero kit parts to show how we accomplish our goal: to work with the Model 3 Performance foundation without disturbing it. The kit now delivers 1896N of downforce, shifting the car from net lift to substantial downforce with a minimal drag increase of just 1.93%. This is achieved through advanced aerodynamic techniques, such as splitter tunnels that enhance front ground effect—features rarely seen on street-legal vehicles. The kit enhances downforce in areas that already have high-pressure air, such as above the grille, over the side inlets, and on top of the front splitter, but also introduces entirely new shapes like the front undertray. Airflow management around the wheels and side skirts is critical for reducing drag and offsetting the added drag from downforce-generating surfaces. Additionally, placing a wing that is highly optimized to minimally disturb Tesla’s refined wake is crucial for maintaining efficiency. While we’re working out a few more final touches with our production run, we are just as excited to reach out to those who have patiently waited and look forward to providing order updates soon!show more