🚨Current scalable RL algos train a policy w/o value... func, which is limiting with learning in open-ended, non-stationary, dynamic environments. But, how to scale value-based RL with more data/compute is unclear... Not anymore: presenting scaling laws for value-based RL 🧵⬇️show more

Does off-policy value-based RL scale? In LLMs, larger scale... predictably improves performance. Value-based RL learns from arbitrary data and is sample-efficient, but folk wisdom says it doesn't scale 🧵⬇️We show predictability for scaling value-based RL!show more

🤔 How to fine-tune an Imitation Learning policy (e.g.,... Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!show more

This figure from HIL-SERL is one of the clearest... visualisations of how RL learns differently from imitation learning. The difference comes down to this: imitation learning treats each (state, action) pair as independent. A correction at timestep 20 teaches nothing about timestep 19 or 21. RL propagates reward backward through time. One successful insertion updates the value estimate of every state along the trajectory. So RL builds a full map of "which states lead to success"; imitation learning just memorizes individual snapshots. Setup: a robot inserting a RAM stick into a motherboard slot. Each dot is an end-effector position (Y = lateral, Z = height). Starting position is randomized. Left to right = training progressing. Top row (RL): the policy builds a funnel. Broad at the top, narrowing into the target. It systematically fills in the state space, learning which paths lead to success from many different starting positions. Bottom row (imitation learning / HG-DAgger, same human data): sparse, diffuse, no funnel. The policy only learns near states the human demonstrated. Both have access to the same data, including human corrections, but a completely different structure emerges.show more

🔥 Nebius AI R&D is hiring AI Research Interns... for short, high-impact RL projects. Exclusive to X right now — no LinkedIn mass postings yet. In 2019, I was a fresh dental grad with 3 months of runway left, begging for an AI shot. I know the grind. We’re looking for sharp early-career folks (students, grads, career-switchers) to join us and work on: > Agent trajectories analysis at scale > Long-horizon tasks for coding agents > Pushing open RL environments > Any other data / RL env / eval project that will benefit open-source community What you get: 💰 Fully paid internship (3-6 month) 📦 100% open-source shipping 📄 Co-author research papers ⚡️ Access to Nebius compute infra 🌍 Remote-friendly (EU/US) or Amsterdam/London/other office. If you’ve done any cool AI/ML/RL stuff, dm me with your most impressive project + 1-sentence summary + cv Sharing appreciated!🤝show more

Introducing RL Environment Creator Skill Now any one can... create RL environments $ npx skills add adithya-s-k/RL_Envs_101 > You can create environments across multiple frameworks like OpenEnv, OpenReward, Verifiers, NemoGym ... > the repo has live working examples of environments that your coding agent can reference > The skill is design to first understand what type of model you are training and create an environment while keeping that in mind ps. There’s a lot more to building RL environments that can be used for training. One major aspect is the data, which this skill can’t directly solve. However, the skill will help with implementing tools, rewards, and other components of an RL environment, making it easier to go from idea to implementation quickly across different frameworks. Let me know if you’d be interested in a detailed, end-to-end blog/tutorial on building an environment and actually training a model for a useful use case.show more

New research from Databricks: LLMs Can Learn to Reason... via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.show more

RL is painfully slow 😭 — bottlenecked by super-long... CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: Code: Blog:show more

AI has had exactly two scaling axes that worked... so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inferenceshow more

March 18, 2025 marked the public launch of OptimAI.... In one year, it has evolved from a lightweight node layer into a decentralized intelligence infrastructure powering real-time data, compute, and reinforcement for agentic systems. Not just nodes. Not just data. A continuously learning, network-driven intelligence layer. This is infrastructure for a new class of software: autonomous agents that persist, adapt, and operate across environments. Year one established the network. Year two is where it compounds into coordination and value flow. Personal agents. Reinforcement at network scale. Emerging primitives for AgentFi. New layers coming online. 2026 won’t just be about scale, it’s where the network starts to operate. Keep building!show more

🚨MARKETS: WATCHERGURU SAYS $150 $XRP IS PRACTICALLY IMPOSSIBLE In... a new article titled 'XRP Has No Future: What the Numbers Really Tell You', WatcherGuru points out that to reach a token value of $150 per $XRP, based on current supply... ... $XRP's market cap would have to reach $13.5 trillion. "That is around 10 times Bitcoin’s current value... This is a big part of why XRP will never go up to the levels that viral social media posts keep promising" The report does however, highlight the various major institutions that leverage Ripple's technology, as well as more feasible price predictions for 2026. This includes 21Shares' estimate for $2.45.show more

Chainlink Now Secures An Insane $110,000,000,000 Chainlink has just... surpassed the $110 billion mark in total value secured. The incredible figure is split across $LINK's cross chain interoperability protocol (CCIP) as well as its data feeds, on the below basis: - CCIP $60+ billion - Data Feeds $50+ billion With more than $30 trillion in transaction value enabled, and not a single day of outflows from the five spot $LINK ETFs in the US, 2026 is shaping up to be a fantastic year for the Chainlink army.show more

There is nothing more valuable than your time. And... yet it is time that most goes to waste. Money can’t give you more time. But money can get you the time of others. And that time is worth much much more than most folks are being compensated in exchange for it. You can’t get that time back. The majority believing there is little value for their time is part of how those who do assign high value to their own time keep their control. Time is a real resource. Money is a social construct. One of the two is much more valuable. But what do I know, I’m just a silly girl with hand puppets.show more

We're excited to unveil NRN Agents, a rebrand that... aligns our project identity with our token and strengthens our mission to power the future of AI-driven gaming. This mission requires collaboration, and starting this week, we will begin our expansion to become a multi-chain ecosystem. We are joining forces with leading gaming platforms and ecosystems to realize this vision. Stay tuned for more announcements to come. Why NRN Agents? NRN stands for NEURON, the fundamental unit of intelligence. Our AI agents function as the neural foundation of games, learning, adapting, and evolving within game worlds to deliver unparalleled engagement. NRN agent SDK enables advanced gaming agents powered by a proprietary machine learning infrastructure focused on behavioral learning. We've perfected the craft of gaming agent design, creating hyper-efficient agents that are performant and scalable—from casual to the most demanding games. Our SDK will seamlessly integrate into many platforms, tech stacks, and ecosystem – Any Game. Any Chain. More than just games, it's the path to AGI Gaming is our proving ground, but not our final destination. We're using games as a sandbox to accelerate the development of generalized intelligence—one that will create meaningful real-world impact. With the upcoming launch of [redacted] and a growing network of partners committed to the AGI vision, we're building an open-source innovation movement powered by an AI x gaming framework connected by $NRN. $NRN the token $NRN is a utility token that serves as the gateway to our growing ecosystem. It will power a diversified economy with multiple revenue streams and staking opportunities: Agent Deployment: NRN is the laboratory creating gaming agents that can be distributed through platforms and launchpads alike. The model is simple: More games integrate, more NRN agents get deployed, more monetization. Data Creation: NRN Reinforcement Learning (RL) enables token staking to create Data Capsules. Players contribute gameplay data into the Capsules, which are used train RL agents and reward participants (players & stakers). AI Arena: $NRN also continues to power AI Arena's in-game economy, a cult favorite of competitive diehards that features a skill-based wagering system. To our community who have supported us since 2021: thank you for being part of our journey—the next chapter will be the most exciting yet!show more

🧬 We have many foundation models or language models... for DNAs, but can we control them? We introduce Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RL — a reinforcement learning framework for controllable cis-regulatory sequence generation. Paper: Code: 🔬What’s the challenge? Designing regulatory DNA that is both highly expressive in target cell types and inactive in others is essential for synthetic biology, gene therapy, and precision medicine. Yet, controlling these trade-offs is challenging due to sparse, sequence-level rewards and biological constraints. 🔥Why Ctrl-DNA? Ctrl-DNA fine-tunes pre-trained DNA language models using a value model free, Lagrangian-guided RL framework, enabling flexible and customizable constraint optimization. Users can define application-specific thresholds across cell types, balancing expression strength with specificity. ✅ Maximize target-cell expression ✅ Constrain off-target activity under user-defined thresholds ✅ Preserve cell-type-specific TF motif structure Benchmarked on human enhancer and promoter datasets, Ctrl-DNA consistently outperforms prior methods, achieving stronger specificity, higher fitness, and more biologically grounded sequence generation — all with direct control over regulatory trade-offs. Shoutout to the PhD students Xingyu Chen (Xingyu Chen ) and Rex Ma (Rex Ma) for their amazing work leading this project!show more

How can a 99% accurate medical test give you... a 9% chance of having the disease if it comes back positive? 🤔 If you are in medicine this is the SINGLE most important diagnostic testing concept to know. Welcome to the difference between specificity and positive predictive value. Sensitivity & specificity are fixed test properties. These do not factor how common a disease is (prevalence) Positive Predictive Value (probability a positive test reflects having a disease) factors in prevalence and is actually more important to clinicians than sens/spec. It is harder to figure out though because we need to have a gestault for how prevalent a disease is for the EXACT patient we are seeing. If you have very low prevalence, even with a great test, most positives are false positives. This is why screening low-risk patients can result in many false positives and harm To master this, just play with the calculator yourself and you will see!!!!👇show more

Imagine OptimAI Data Network is a giant library that... AI Agents use to learn and work. 📚 But here’s the twist, instead of one person deciding what goes in the library, everyone in our community can help pick, check, and improve the books (data). That’s what OptimAI DataDAO is: + DAO? It stands for "Decentralized Autonomous Organization", fancy words for a club where we all make the rules together, no single boss in charge. Like a playground game where kids vote on the fun! + It’s how we, the community, decide together which data is accurate, useful, and fair for AI to use. + The more you contribute, the stronger our network gets, and the more value we all share. You're building the future! Stay tuned - we’re building something that will change how AI learns. BUIDL with us:show more

Math quant bot on Polymarket made over $457K PnL... in 20 days - he turned $7,387 → $457k profit it uses Markov Chains to find "mispriced" windows on BTC and ETH up/down markets - made 14,200+ predictions, with ~$22,850 avg. daily profit by exploiting gaps humans miss at 3AM strategy: a 1h BTC/ETH up/down market is a binary contract it pays $1 if event happens → $0 if not Markov Chains give you the probability of the next market state based on: > current state of the market (up / down / flat) > transition matrix built from live price data > diagonal persistence value - how stable the current state is formula: p̂ − market_price ≥ 0.05 AND P(j*, j*) ≥ 0.87 bot profile: - read article below to understand how Markov Chains are used to extract edge from prediction marketsshow more