Thanks AK! Finally, robot can do continuous, agile, autonomous,... adaptive jumping over stair and stepping stone Key idea: combine the pros of model-free RL and model-based control. RL (for CoM refs) + QP (for GRF) + WBC (for torque) Open-sourced:show more

GPT-5.5 by Reasoning Effort: I've asked it in Codex... to create a physics-based visualisation of RL cycles for different sized models (70b, 1t, 10t), to demonstrate how the amount of RL you can do differs by model size. My assessment of each: - Low: weird slop - Medium: kinda cooked - High: sort of tried but ultimately incoherent - Extra High: elite - really nice idea and well executed Obviously this is just one shot, but worth trying different reasoning levels for the new models, medium seems to be pretty good for GPT-5.5 and it was really bad for many previous GPT models.show more

New project! Flow Policy Gradients for Robot Control tldr;... a simple online RL recipe for training and fine-tuning flow policies for robots co-led w/ Hongsuk Benjamin Choi:show more

Does off-policy value-based RL scale? In LLMs, larger scale... predictably improves performance. Value-based RL learns from arbitrary data and is sample-efficient, but folk wisdom says it doesn't scale 🧵⬇️We show predictability for scaling value-based RL!show more

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning... Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fangshow more

Excited to share CrystalReasoner, a reasoning model for crystal... structure generation with LLMs and property-conditioned generation through RL: Website: Paper: Code:show more

Intense battle! A crucial opportunity for number 3! #bipedal... #Robot #RLshow more

Why am I working on RL for LLMs, when... I can go work on RL for Olaf... #humanoidssummitshow more

Introducing RL Environment Creator Skill Now any one can... create RL environments $ npx skills add adithya-s-k/RL_Envs_101 > You can create environments across multiple frameworks like OpenEnv, OpenReward, Verifiers, NemoGym ... > the repo has live working examples of environments that your coding agent can reference > The skill is design to first understand what type of model you are training and create an environment while keeping that in mind ps. There’s a lot more to building RL environments that can be used for training. One major aspect is the data, which this skill can’t directly solve. However, the skill will help with implementing tools, rewards, and other components of an RL environment, making it easier to go from idea to implementation quickly across different frameworks. Let me know if you’d be interested in a detailed, end-to-end blog/tutorial on building an environment and actually training a model for a useful use case.show more

🤖 Another zero-shot reward model is now in LeRobot:... ROBOMETER. A general-purpose, zero-shot video-language reward model from University of South Carolina, UT Dallas, Massachusetts Institute of Technology (MIT), University of Washington, Ai2, and NVIDIA that predicts frame-level task progress. Trained on 1M+ trajectories from 21 robot embodiments, generalizes zero-shot to unseen tasks, scenes, and robots. 2.4–4.5x better downstream success rates across online RL, offline RL, data filtering, failure detection, and data retrieval for IL. Project: Paper:show more

INSANE...this is an Open Source Video model available for... free on Hugging Face. LongCat just dropped an amazing video avatar model that can do crazy things. And it's 100% Opensource.show more

I've gotten a mujoco sim RL training loop for... a unitree robot at 200k SPS. I'm looking into the physics for friction, contact dynamics. My goal: can I reproduce & beat the mujoco playground RL baselines This is running in my web browser with raylib. Its the baselineshow more

AI agents can now control robots! For ClawCon, we... integrated OpenClaw🦞 and Robot Operating System (ROS) - the largest open-source robotics stack powering millions of robots worldwide. If you had an autonomous agent IRL, what would you make it do?show more

My Model Y today on FSD V14.3.2 slowing down... for a small bunny in the middle of the road. In the V14.3 release notes, Tesla specifically says they “improved handling of small animals by focusing RL training on harder examples and adding rewards for better proactive safety.”show more

GR-RL Going Dexterous and Precise for Long-Horizon Robotic Manipulation

Zombie robot RL policy

1/ NVIDIA just open-sourced Cosmos 3 at GTC Taipei!... It's the first fully open "omnimodel" for physical AI - one model that understands the real world, predicts what happens next, and generates the actions a robot should take. Weights, code, datasets. All open. And this is really big. Lets dig into everything: 🧵show more

🚨Epic News: Demos for ExVideo and Diffutoon are out!... 🤯 Play with the advanced parameters for greater control -- this might be the best open model and demo for text2video/image2video out there!show more

We are excited to launch our two models Pharia-1-LLM-7B-control... and Pharia-1-LLM-7B-control-aligned. Both models and the code used to train them are now publicly available and open-sourced for non-commercial research and educational use. Read our model blog post here: Learn more about our open-source codebase Scaling: #writtenbyalephalphashow more

RL is painfully slow 😭 — bottlenecked by super-long... CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: Code: Blog:show more

🤔 How to fine-tune an Imitation Learning policy (e.g.,... Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!show more