New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL
Databricks AI Research
Visit on X@DbrxMosaicAI55K followers
12,860 views
New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL
Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning.
It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations.
For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.
Comments
Comments from the original post on X (Twitter)
No comments yet
