Name: Reinforcement learning should be able to improve upon behaviors seen when training. In practice, RL agents often struggle to generalize to new long-horizon behaviors. Our new paper studies *horizon generalization*, the degree RL algorithms generalize to reaching distant goals. 1/
Uploaded: 2025-02-04T19:14:25.000Z
Duration: PT15.933S
Channel: Vivek Myers
Description: Vivek Myers shorts video about Reinforcement learning should be able to improve upon behaviors seen when training. In practice, RL agents often struggle to generalize to new long-horizon behaviors. Our new paper studies *horizon generalization*, the degree RL algorithms generalize to reaching distant goals. 1/

Reinforcement learning should be able to improve upon behaviors... show more

Vivek Myers

79,523 Aufrufe • vor 1 Jahr

RL X-mas came early. 🎄 For too long, building... show more

Weights & Biases

112,643 Aufrufe • vor 9 Monaten

Haven't been to a conference in a while, really... excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)show more

Abhishek Gupta

10,802 Aufrufe • vor 1 Jahr

RL is painfully slow 😭 — bottlenecked by super-long... CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: Code: Blog:show more

Infini-AI-Lab

77,519 Aufrufe • vor 28 Tagen

RL is back! But is it always the best... show more

Sebastian Risi

11,130 Aufrufe • vor 11 Monaten

New research from Databricks: LLMs Can Learn to Reason... via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.show more

Databricks AI Research

12,539 Aufrufe • vor 4 Monaten

Does LLM RL post-training need to be on-policy?

Kianté Brantley

113,605 Aufrufe • vor 4 Monaten

Language models understand natural proteins. But can they generalize... show more

Tom Sercu

275,213 Aufrufe • vor 3 Jahren

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical... Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.show more

Robora

23,489 Aufrufe • vor 9 Monaten

🔬🤖Excited to share our new preprint on robust virtual... staining of key organelles! ✨ Virtual staining and segmentation of nuclei and cell membranes in phase images can significantly accelerate image-based screens and developmental studies. 🔍 Current virtual staining models often struggle with nuisance variations in label-free input and don't generalize well to new cell types. We’re releasing our training protocols that lead to robust models and enable few-shot generalization. 🔧 Stay tuned for our revamped codebase with weights and example data, coming soon! 👏🧫🐟 Huge shoutout to Ziwen Liu, Eduardo Hirata-Miyasaki, my team Chan Zuckerberg Biohub Network (Soorya, Johanna, Christian, Talon Chandler, Ivan Ivanov), and the teams of Manuel Leonetti, Carolina Arias, and Adrian Jacobo for this fun collaboration! #VirtualStaining #DL #CellBiology #Zebrafishshow more