Introduce EAGLE, a new method for fast LLM decoding... based on compression: - 3x🚀than vanilla - 2x🚀 than Lookahead (on its benchmark) - 1.6x🚀 than Medusa (on its benchmark) - provably maintains text distribution - trainable (in 1~2 days) and testable on RTX 3090s Playground: Blog: Code: ⚒️First Principle: Compression! Yi Ma We find that the sequence of second-top-layer features is compressible, making the prediction of subsequent feature vectors from previous ones easy by a small model. 🙏Acknowledge: This project is greatly inspired by the Medusa team (Tianle Cai @yli3521 Zhengyang Geng Hongwu Peng Tri Dao), the Lookahead team (Hao Zhang LMSYS Org), and others. Joint work with Yuhui Li and Chao Zhangshow more

Jointly announcing EAGLE-3 with SGLang: Setting a new record... in LLM inference acceleration! - 5x🚀than vanilla (on HF) - 1.4x🚀than EAGLE-2 (on HF) - A record of ~400 TPS on LLama 3.1 8B with a single H100 (on SGLang) - 1.65x🚀in latency even for large bs=64 (on SGLang) - A new scaling law: more training data, better speedup - Apache 2.0 Paper: Code: SGLang version: ⚒️Takeaway: Introducing training-time test, a novel draft model training technique: we replace feature prediction with direct token prediction and shift from top-layer-only features to multi-layer feature fusion. This approach unlocks a new scaling law previously undiscovered in EAGLE and EAGLE-2. 🙏Acknowledge: We would like to thank the SGLang team (zhyncs Lianmin Zheng Ying Sheng James Liu, Ke Bao, and others LMSYS Org) for their merge and careful evaluation of EAGLE-3 on SGLang. 🤝Want to collaborate? We're a small academic group with limited GPU resources. If you're interested in supporting our next version of EAGLE or would like us to train a preliminary version tailored to a specific model, please get in touch! Joint work with Yuhui Li, Fangyun Wei, and Chao Zhangshow more

🚀 A better, faster co-folding-based binding affinity model. Predicting... how tightly a drug candidate binds to its target is critical in drug discovery. It also requires massive computational resources. State-of-the-art models can take 20 seconds to a minute per prediction, impractical for the demands of large scale early-stage programs . 💠 Today, Recursion’s Valence Labs is releasing Nesso-1: the fastest open-source co-folding-based binding affinity model available. At 1 second per prediction, it’s roughly 20x faster than our previous collaboration on Boltz-2 while matching or surpassing its accuracy across public and internal benchmarks. By leveraging NVIDIA Healthcare cuEquivariance, we’ve been able to further accelerate both training and inference by an additional 2-3x. We look forward to continuing to improve Nesso-1 in collaboration with NVIDIA. Weights and code are fully open-sourced. The core architectural ideas behind Nesso-1 build on the insight that coarse-grained co-folding representations can match full-atom models for affinity prediction at a fraction of the cost. Nesso-1 is the first open implementation of this approach with no proprietary dependencies, trained entirely on public data, built to be reproducible and extensible. We’re already using Nesso-1 internally in active drug discovery programs. Fast, reliable affinity prediction at scale is foundational to the kind of autonomous design loops that define our vision for Autonomous Precision Design and Nesso-1 is a meaningful step toward that. 👉 Report: 👉 Github: 👉 HF:show more

Prediction Market is one of the leading Web3 niches... in 2024! With about $4 Billion in trading volume, $192 Million in TVL and millions of users in 2024, prediction protocols have been showing tremendous growth and attracting user adoption in Web3. PolyMarket seems to be leading the pack, with over $175M in TVL, and backing from Vitalik Buterin. However, a majority of Prediction Markets, including PolyMarket, currently lacks the flexibility and capital efficiency required for seamless transactions. Also, they are all majorly focused on driving Web2 users thus neglecting the need for markets that cater for short-term, high-risk investments from Web3 Degens. To tackle these flaws, there's a need for a revolutionary contender that understands the need of Web3 Chads That's where Predict Hub comes in PredictHub is a prediction Market that transforms real world events into opportunities for everyone to participate and forecast. Launching on Arbitrum, PredictHub is already catching the attention of major players in the Space by offering something PolyMarket and others don't - Flexibility and Incentives. By offering fast market updates and innovative prediction category like ETF Forecasts, PredictHub is changing how we interact with Prediction Markets. But then, here's where it gets more interesting; PredictHub offer users a unique point system, where you don't just make predictions, you also earn rewards. The more you Predict, the more you earn. These rewards are 2-fold: Nova and Orbit Points. Nova Points are earned by traders based on their trading activity and their leaderboard ranking. Orbit Points, on the other hand, are earned by users who provide liquidity, based on their LP size and duration. Other Point systems include PolyMarket User Points, Leaderboard Bonus and Market Multipliers. These rewards offer users more competitive edge than other prediction markets. Apart from these rewards, PredictHub features a unique 3-tier referral system, rewarding users with even more as you invite your friends. The more friends you bring, the greater the rewards. On top of these, PredictHub focuses on USDC and a wide-range of yield bearing assets like GLP, gUSDC, and sUSDe, enabling users to optimise their earning while holding assets across Networks. Exciting, right? PredictHub is in its Testnet phase and you can start earning Points Right away 🔅 Here's how to Get Started on PredictHub: 1. Go to 2. Request Faucet 3. Start making predictions and earning Points Easy-Peasy ✅ More Info can be gotten from Predict Hub All eyes are on PredictHub as the fix for the flaws of Prediction Market Protocols. With its unique approach targeting untapped niches that most existing prediction markets have yet to explore, I believe the Protocol has the potential to become a breakout success I will be placing good Predictions to Position 🚀🚀🚀show more

Getting from $1M to $100M in ARR took us... 18 months. The second hundred took less than six. Last week Legora passed $200M. 130,000 lawyers now use Legora on a monthly basis, across 2,100 firms and legal teams in more than 80 countries. Two things from this year I would not have predicted: The first is how fast the US moved. 18 months ago we had no office in North America. Today over half of the AmLaw 50 work with us, out of our offices in New York, Denver, Chicago, Houston, San Francisco and Toronto. The US is our largest market by revenue. The second is where the demand is coming from. More than 40% of our new customers are now in-house legal teams, including Air Canada, Salesforce and Palo Alto Networks. Two thank-yous, before we get back to work. To our customers, who backed a small company from Stockholm with a lot to prove, and who still tell us plainly when we get something wrong. And to the team, engineers and lawyers sitting next to each other, who turn that into something better, faster than any group I have worked with.show more

We’re excited to introduce Text-to-LoRA: a Hypernetwork that generates... task-specific LLM adapters (LoRAs) based on a text description of the task. Catch our presentation at #ICML2025! Paper: Code: Biological systems are capable of rapid adaptation, given limited sensory cues. For example, our human visual system can quickly adapt and tune its light sensitivity to our surroundings. While modern LLMs exhibit a wide variety of capabilities and knowledge, they remain rigid when adding task-specific capabilities. Traditionally, customizing these models requires gathering large datasets and performing often expensive, time-consuming fine-tuning for specific applications. To bypass these limitations, Text-to-LoRA (T2L) meta-learns a “hypernetwork” that takes in a text description of a desired task, as a prompt, and generates a task-specific LoRA that performs well on the task. In our experiments, we show that T2L can encode hundreds of existing LoRA adapters. While the compression is lossy, T2L maintains the performance of task-specifically tuned LoRA adapters. We also show that T2L can even generalize to unseen tasks given a natural language description of the tasks. Importantly, Text-to-LoRA is parameter-efficient. It generates LoRAs in a single, inexpensive step, based solely on a simple text description of the task. This approach is a step towards dramatically lowering the technical and computational barriers, allowing non-technical users to specialize foundation models using plain language, rather than needing deep technical expertise or large compute resources.show more

We’re excited to introduce ShinkaEvolve: An open-source framework that... evolves programs for scientific discovery with unprecedented sample-efficiency. Blog: Code: Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources! Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful. On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a big leap in efficiency compared to previous methods that required thousands of evaluations. We applied ShinkaEvolve to a diverse set of hard problems with real-world applications: 1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency. 2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition. 3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity. ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job. By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it! Learn more in our technical report:show more

Ok. I’m so excited to announce a bunch of... new things that I don’t even know where to start. Finally wrapped up a MAJOR update for EasyWaterscape! My biggest one ever, and the fruits of nearly 2 months of work so far. 1. HUGE performance gains. The per-frame cost is cut by about 67%, making EWS nearly 3x faster than before. 2. A revamped water mesh that runs better AND looks better. It’s optimized for TSR (far less ghosting/smearing), and gives you fine, granular control over the exact triangle density you need (or can afford) for your project. This makes EWS far more suitable for games than it was. 3. A brand new documentation site with info on all the new features, the changelog, the new benchmark results and more: 4. Improvements to Underwater (BETA), including water surface meniscus. I can't wait to spend more time on the underwater side, it's going to look fantastic when I'm done with it. (Edge blur & distortion aren't in yet, but they're coming.) Man. This has been so satisfying and a ton of fun. #DellTech #NVIDIA #DellProPrecision #unrealengine #gamedevshow more

Transformer by hand ✍️ ~ 6 steps walkthrough below... Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearningshow more

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with... supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)show more

Building The On-Chain Cooperative 🟡 Welcome to the dawn... of a new era in the crypto space, where the buzzword "community" is not just a hollow echo but a vibrant force that propels us towards a brighter future. Let's delve into the heart of MODE, the Onchain Cooperative that seeks to redefine the landscape of web3. What does MODE stand for? MODE stands for building an on-chain cooperative focused on sustainable growth and collective prosperity. At its core, MODE is guided by the principles of cooperation, shared incentives, and community-driven development. The goal is to shift from the "fat protocol" mentality where most value accrues to the blockchain/protocol itself, towards an ecosystem where builders, users, and applications can thrive together. What’s MODE's vision and mission in the web3 space? MODE's vision is to return to web3's founding promise - a future that is better for all, not just the individual. A world with aligned incentives that drive growth for everyone involved. A place with opportunities for all, not just the few. The mission is to pioneer the on-chain cooperative - where contributors are rewarded fairly based on the value they provide. Features like Sequencer Fee Sharing distribute a portion of fees to smart contract developers, incentivizing participation. The aim is to encourage collaboration instead of confrontation. Together, the MODE community can deliver new models for cooperation and shared prosperity in web3. Mode Network will solve many problems today in Web3: • Lack of incentives for developers: Developers creating decentralized apps (dApps) currently have few direct economic incentives to create and maintain their projects. Mode provides them with a steady source of income through fee-sharing. • Lack of collaboration: There are few incentives for blockchain projects to compete less and collaborate more for the benefit of the entire ecosystem. Mode's model encourages collaboration by aligning participants economically. • Excessive value accrual at the protocol layer: Mode aims for a more balanced model where the protocol's success is fueled by the success of application developers/builders and the wider community. Growth is a two-way street – "as we grow, you grow". The MODE Pledge 💛 The promise of crypto and blockchain is a brighter future. One that is better for all not just the individual. Where nothing is more important than community. We've strayed from this path. Entering a world of player vs player. Where value is extracted rather than shared. The game is zero sum rather than positive sum. And incentives are aligned with domination, rather than cooperation. Mode is the dawn of a new age. and a return to the promise of what can be. A world with aligned incentives that drive growth for builders, users and projects. A place with opportunities for all, rather than the few. Where we say goodbye to the 'fat protocol', and hello to the onchain cooperative. Join us on our mission to grow together. If this vision for a community-powered web3 ecosystem resonates - where creators are rewarded for their contributions - you can join the MODE on-chain cooperative! Visit Join the discord community Follow Mode 🟡 Together, we can transform web3 into a positive-sum game that unlocks new possibilities for all. Where your growth fuels the growth of others. Let's build the on-chain cooperative!show more

BREAKING: Anthropic just dropped Opus 4.8—and it is a... MONSTER We've been testing for about a week Every 📧 and our verdict is they could've just called it Opus 5, it's that good. Here's our vibe check: - Beats GPT-5.5 on Senior Engineer bench. On our toughest benchmark Opus 4.8 scores a 63—a hair higher than GPT-5.5's score of 62, and a full 30 points higher than Opus 4.7. It tackled a ground-up rewrite of a production codebase, and actually built something that works. HOWEVER: Coding performance varied a lot at different reasoning levels. We recommend using it on xhigh for best results. - Incredibly good writer. Opus 4.8 scored a 79.6 on our writing benchmark—measuring models on real-world writing tasks we do all of the time like essay writing, promo email writing, and more. It beats GPT-5.5 by 6 points. It produces well-written prose with fewer "AI-isms". It's also very good at writing in your voice given the right context. HOWEVER: Writing performance also varied with reasoning levels. Medium reasoning had higher incidence of AI-isms—we found best results with high. - Beast at knowledge work. Opus 4.8 is very good at general knowledge work tasks like report creation, research and more. It produced the best PowerPoint one-shot we've ever seen on our deck generation benchmark. - Emotionally intelligent, willing to question the frame. I've also found it to be quite good at talking through psychological or interpersonal issues. It has a high EQ, and it's also good at not glazing and helping to expand your perspective. Its thought process feels extremely rich and dynamic. THE BAD: These days a model is only as good as its harness, and Codex is still a far superior harness to the Claude Desktop app. This has kept me using Codex + GPT-5.5 as my daily driver, but I am flipping back and forth a lot more between Codex and Claude. Anthropic is back baby! Read the rest on Every 📧:show more

1 Neural Network + Obsidian + Karpathy’s 1-file method... = the most unhinged second brain build of 2026. It remembers everything you’ve ever done, and it costs $0 on top of what you already pay. The base is Karpathy’s append and review: 1 giant note, new thoughts stack on top, old ones sink, every few days you reread and pull the survivors back up. No folders, no tags, no plugins the rereading IS the system, because review is what turns storage into thinking. The flaw: past 10,000 lines, no human rereads anything. That’s where the neural network takes over. You keep the note in Obsidian 1 vault, everything dumps to the top: ideas, links, meeting fragments, half-thoughts. You never organize, you only dump. It all lives as plain markdown on your own disk, and that detail is the whole trick. Because now you point Claude Code at the vault folder, and it reads every line you’ve ever written. “What did I think about pricing in March.” “Find the 3 ideas I keep circling.” “What did I drop that deserves a second look.” It answers from YOUR notes, with quotes, in 15 seconds. Then once a week, 1 prompt closes the loop: read the last 7 days, surface the 5 entries worth pulling back up, flag anything that contradicts what I wrote a month ago. The model does the sinking and surfacing Karpathy did by hand, and the note stays alive instead of turning into a graveyard. Week 1 feels like nothing. Week 4 you hit the first “I already solved this in January.” Month 3 you consult your past self more than Google. Most second brains die in 11 days under 40 plugins and 200 folders. This one is 1 file and a loop, and it compounds because dumping takes 0 discipline. Notion stores what you thought. This thing argues back.show more

Don't train the model, evolve the harness. I read... a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.show more

Had an awesome time with Mark Haywood and the... $edm.v team on the site tour! The DFO permit is a near term major catalyst, which upon receiving will be followed by the final production decision. The water at the polishing pond after the tailings pond is astoundingly clean and loaded with fish, and tested as much cleaner than the water of the local streams and rivers. DFO permits are very rarely not granted, and $EDM.v's thoroughness and hard work is sure to be rewarded. The provincial government of Nova Scotia is also very supportive of the project and is eager to see them enter production. EDM's cash flow for next year will likely be greater than the current mcap, only to continue ramping up to double or greater than the current mcap. Very few juniors are this close to cash flow with such a small capex needed for restart, which will be financed with debt and paid back in around half a year. $edm.v should be 3-4x higher now, at 0.5x NPV, and even higher once the DFO permit is received this summer, 0.7-0.8x NPV. Right now we are trading near 0.10-0.15x NPV when you adjust for the DMS update, current metal prices, as well as doubled Gypsum prices, which is crazy cheap for how advanced the project is. Another impression from the tour is how solid and well made the 2700tpd mill is. It was built by Imperial Oil, a multi billion dollar company, who put in a lot of extras. There are about 5-10m CAD in replacement and maintainence parts on site from the previous opperators, still in original packaging. All of this is just the base case projection, it doesn't include the $gold optionality. Once the mine is in production and the main put is drained, they can go back to where the last producers will pulling ore and resuming mining high grade gold, which left off at 20k to 60k ounces a year based on a rough back calculation from the left over concentrates. This gold is proven via production and recovery, its absolutely there, and the team is actively working on defining the extent and quantities of it. I have been adding to my position this week $zinc $lead $gold $silver $gypsum $emo.v $fwz.v $teck $nexa $glenshow more

introducing a new, very fun, LLM benchmark- the Game-of-Life... Bench! the rules are simple: given an 8x8 grid following Conway's game of life rules, the goal is to create an initial pattern with at most 32 cells that can last the longest number of turns before dying/repeating. some results to highlight (with caveats detailed below): - gpt 5.1 lasts the longest with a 106 step run - claude models are really bad at this! they refuse to reason about this task and score < 25 points - deepseek r1 is the best open model with 102 steps. why? because i wanted to create a benchmark that has (i think) no practicality, but is still fun to look at, cheap, and still measures something interesting. i also am a big fan of the game of life. its absurdly simple rules leading to intractability is extremely cool to me. also, i saw a lot of work with LLMs trying to "predict" the next state in Conway's game of life, I think game-of-life bench is more fun because it's pretty open ended and only asks the LLM for the initial state. I also think this could be an RL env? but idk why you would ever train on this task haha i don't think this is a "serious" benchmark because it doesnt measure anything practical, but i still think it's a hard benchmark exactly because you can't predict what happens with your initial state many turns into the future; this is why i was initially expecting all LLMs to be bad at it, but turns out, some are clearly better than the others (the ordering may surprise you!) reminder: this is still a work-in-progress; (1) i am gpu-poor so could only do 10 runs for each model, even though total running cost is relatively low. maybe with some more credits i can run more seeds for each model. (2) i handpicked models which i think are at the frontier right now, plus some others that were on my mind. so, if you'd like to see a model on here, let me know. (3) i currently only do an 8x8 grid because i thought that by itself would be pretty hard for current LLMs, but of course we can increase grid sizes! (4) the coolest thing is, i dont think we can calculate the max possible number of states (yay undecidability!) you can go without repeating, so this is essentially a no-ceiling task, which is pretty cool! again, i did this mostly out of a desire to make LLMs do something fun. if this keeps me entertained for a few more days, i'd likely release a blog post on it. if it keeps me entertained for a week (and someone sponsors me), i'll put more work into it :P lastly, this is fully open sourced, so feel free to run this on your own!show more

🚨 RWA PAD PLATFORM LIVE AND FIRST PROJECTS COMPETE... ON DEMODAY 🚨 Hello EstateX Family, RWA Pad platform is live. Because of the tech teams occupation there are still some changes made to the website and the content is not completely final. To be able to participate in the projects that will be coming on RWA Pad, you need to hold or lock $ESX. The following tiers qualify, with Tier 5 having first access AND getting the best deal. Tier 5: Unicorn Club (1M+ tokens) Tier 4: 500,000 tokens Tier 3: 100,000 tokens Tier 2: 50,000 tokens Tier 1: 10,000 tokens This also means that new investors / their community need to buy $ESX to participate. 🚨 DEMO DAY ON THE 17TH OF FEBRUARY On the 17th of February, the first projects will compete for a spot on the launchpad. These projects will bring their audience, and judges will be our (famous) project partners and KOLs. On top of that, the EstateX Family will have a decisive vote on the projects that they want to see on the launchpad. In a battle style form, they will compete and create content, viral moments and bring audience. A week later the first raise will happen on RWA Pad, followed by buyback of the revenue generated. No matter what happens, the team keeps showing up. On Monday we have the L1 chain Beta launch with a Graham AMA, we have Sky Villa’s opening up, other property payments opening up with continued sales, the demo day, RWA Pad raise with all followed by generated revenue buybacks. Make sure to watch today’s AMA as we also discussed price action, the team, the switching of legal framework to make property sales more efficient (hence the delay), how the team moves forward and what’s upcoming. 🚨 REGISTER FOR YOUR TIER ON RWA PAD You can register for your tier on RWA Pad now, by selecting one of the five. Click register now on the site below. *Content is not finalized yet and subject to change. We are also curious if you want to see other sectors apart from RWA (like AI, perps, prediction markets, privacy tokens or anything else). Let us know below 👇show more

NEW WORLD MODEL: Yann LeCun's team is back with... an efficient model! This project involves Yann LeCun, Lukas Kuhn, Lucas Maes, Quentin Le Lidec, and Randall Balestriero. A couple definitions first: - DINO: self-DIstillation with NO labels. A self-supervised image model (Meta, 2021) where a student network learns to match a teacher (an EMA copy of itself) across two crops of the same image, with no labels and no negatives. - SIGReg: a regularizer that prevents embedding collapse by forcing the embeddings to match an isotropic Gaussian, tested with a normality test (Epps–Pulley) on many random 1-D projections instead of in full dimension. LeVJEPA is a self-supervised video pretraining method, released with open code, weights, and checkpoints. It learns a video representation by pushing the embeddings of global and local crops of the same clip together (an invariance loss), while a regularizer called SIGReg forces the embeddings toward an isotropic Gaussian to provably prevent representation collapse. Unlike V-JEPA and V-JEPA 2 it uses a single shared encoder with a projector and no target network, no predictor and no stop-gradient. It drops 95% of tokens per view, uses block-causal attention (each frame attends only to past frames), and has a single loss weight. It is evaluated purely as a representation learner via frozen probing on ImageNet-1K, Something-Something-v2 and Kinetics-400, not on any robot. What I find interesting, is that V-JEPA and V-JEPA 2 need an EMA target encoder, stop-gradients and a capacity-limited predictor to avoid collapse; LeVJEPA drops all of it for one shared encoder plus projector, preventing collapse instead with the SIGReg regularizer under a provable guarantee and a single hyperparameter. The "P" (predictor) in JEPA is effectively gone. LeVJEPA is also less compute intensive: - 5.6x to 20.8x lower total pretraining compute than V-JEPA 2 - 7.6 points higher on ImageNet-1K at matched FLOPs - trains at batch size 128 within 8GB where V-JEPA 2 saturates at batch size 28 Also worth mentioning: ImageNet-1K accuracy rises monotonically with the token-drop rate, from 33.9% at rho = 0 to 47.6% at rho = 0.95. The aggressive dropping is actually doing regularization work. On the JEPA-versus-DINO debate: - it loses to DINOv2 by 3.1 points on ImageNet-1K (appearance, static) - but wins on Something-Something-v2 by nearly 2x (motion, temporal) - and beats V-JEPA 2 by 1.9 points on ViT-L at 5.6x lower cost. -> optimized for temporal and motion understanding per compute dollar.show more

we sped up distributed inference by up to 5x... with decentralized speculative decoding. many don't realize that AI models normally generate text one single word at a time, waiting for the network after every word. speculative decoding changes this by using a "guess & confirm" system, similar to autocomplete. how it's done: 1. draft locally (the guess) instead of waiting for the network, a tiny, fast model on your device guesses the next few words instantly, without waiting for the network. 2. confirm remotely (the check) the massive remote model doesn't generate from scratch; it just checks the draft. it looks at the guesses in a batch and says "yes, yes, no." you get multiple words in the time it usually takes to get one. 3. adaptive logic dsd is smart. if the topic is creative, it lets the draft flow loose. if the topic is math or code, it checks more strictly. it balances speed and precision automatically so your inference almost feel instant. find out more: paper: blog:show more

I spend my days explaining to teams why a... 770-billion-parameter open-weight model will never fit into their infrastructure. This week, I asked it to code a complete mobile game from a single prompt. The concept is one everyone has probably played before. A hole moving through an open-plan office, viewed from above, swallowing everything in its path. You start tiny, only able to swallow pens and cups. You grow, moving on to keyboards and plants, then chairs and printers, then desks and vending machines. Eventually, you swallow the entire meeting room. 60 seconds on the clock. One HTML file, zero external libraries, zero errors on launch. The video shows the generation and then the actual gameplay. What broke is more instructive than what worked. The structure came out right on the first try: fixed-timestep loop, spatial grid for collisions, tier system, spring camera. The balancing and rendering, not so much. The first version scored 70 points in 13 seconds with a tier threshold at 500, and drew colorful circles and triangles instead of furniture. I had to give it numbers and exact recipes. Speed: 640 pixels per second. Radii: 30, 58, 96, 150, 225. Tier thresholds: 150, 550, 1500, 3500. And for every object, a pixel-perfect drawing recipe. Once I gave it that, it followed the instructions exactly. A model that doesn't execute its own code won't tune itself. But it will execute, down to the exact numbers, what you tell it to build. The model is Hy4 preview, released by Tencent Hunyuan on August 28. 770 billion parameters in total, but 49 billion active per token. And that second number is what determines your serving bill. Native 1M context. Apache 2.0 license. vLLM and SGLang supported from day one, with an official FP8 checkpoint. Text-only preview. The part that matters for deployment is the compression. Tencent describes it as seven times smaller with almost no loss. GGUF builds use mixed per-layer quantization, where calibration data determines the bit width layer by layer. Some layers go as low as 1.31 bits, while others go up to 2.06 bits, averaging 2.38 bits per weight. The model drops from 1.5 TB in BF16 to 213.66 GiB while, according to their measurements, staying in the same performance range on real-world tasks. On this build, they report 204 tokens/s in prefill and 20 tokens/s in decoding, measured on an 8-GPU node. Their numbers, not mine. Their blind evaluation scores 2.99 out of 4 across 203 engineering tasks rated by 163 experts. Ahead of Kimi K3 at 2.94 and GLM-5.3 at 2.92. Their numbers too. My run went through the official hosted studio, not a local build, so I’m not claiming to have benchmarked the compressed GGUF myself. Two honest caveats. None of these builds run on standard llama.cpp. The hyv4 architecture isn't upstream yet, so patches are required. And 214 GiB of resident weights is still a server, not your laptop. This is a preview, and Tencent explicitly asks users to break it and report what fails. So here's my contribution.show more