Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We can learn a model that provides shaped "process rewards" for robotic RL, that evolves automatically as the policy gets better. This improves performance on benchmarks, and works in the real world! Some fun new work with Raymond Tsao & Andrew Wagenmaker

Sergey Levine

131,801 subscribers

36,440 views • 19 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Building upon SimpleVLA-RL, we have implemented real-world RL on long-horizon dexterous tasks and witnessed a non-trivial (~relatively 300%) performance improvement over the SFT model, along with surprising capabilities on auto-recovery. Blog coming soon. The entire process uses very little data and training compute—basically costing no more than a single robotic arm—hinting that real-world generality for machines is actually within sight.

Building upon SimpleVLA-RL, we have implemented real-world RL on long-horizon dexterous tasks and witnessed a non-trivial (~relatively 300%) performance improvement over the SFT model, along with surprising capabilities on auto-recovery. Blog coming soon. The entire process uses very little data and training compute—basically costing no more than a single robotic arm—hinting that real-world generality for machines is actually within sight.

Ning Ding

93,076 views • 6 months ago

RL in the real world presents some big challenges, but also some really big opportunities. In our new work, HIL-SERL, Charles Xu, Jeffrey Wu, Jianlan Luo show that real-world RL can learn a huge range of precise and robust tasks, and perform them much faster than imitation.

RL in the real world presents some big challenges, but also some really big opportunities. In our new work, HIL-SERL, Charles Xu, Jeffrey Wu, Jianlan Luo show that real-world RL can learn a huge range of precise and robust tasks, and perform them much faster than imitation.

Sergey Levine

36,489 views • 1 year ago

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

AI at Meta

309,942 views • 1 year ago

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia @ RSS 2026, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia @ RSS 2026, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!

Sergey Levine

29,139 views • 12 days ago

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.

Physical Intelligence

705,425 views • 8 months ago

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by Dibya Ghosh, we show how this can be done:

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by Dibya Ghosh, we show how this can be done:

Sergey Levine

48,747 views • 1 year ago

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning *beyond* just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning beyond just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

Abhishek Gupta

13,634 views • 1 year ago

🦋 Last week, our team took delivery of a new quadruped robotic dog from Unitree — the Unitree Go 2 AIR. With Letta and the Animus framework, developers can build embodied agents that think, learn, and act in the real world. Watch how we plan to bring Animus to life on this new hardware. Build 🤖 on Animus

🦋 Last week, our team took delivery of a new quadruped robotic dog from Unitree — the Unitree Go 2 AIR. With Letta and the Animus framework, developers can build embodied agents that think, learn, and act in the real world. Watch how we plan to bring Animus to life on this new hardware. Build 🤖 on Animus

Animus

33,079 views • 8 months ago

Can we build a generalist robotic policy that doesn’t just memorize training data and regurgitate it during test time, but instead remembers past actions as memory and conditions its decisions on them?🤖💡 Introducing SAM2Act—a multi-view robotic transformer-based policy that integrates a visual foundation model with a memory architecture for robotic manipulation. Project page: 🧵👇

Can we build a generalist robotic policy that doesn’t just memorize training data and regurgitate it during test time, but instead remembers past actions as memory and conditions its decisions on them?🤖💡 Introducing SAM2Act—a multi-view robotic transformer-based policy that integrates a visual foundation model with a memory architecture for robotic manipulation. Project page: 🧵👇

Jiafei Duan

87,573 views • 1 year ago

The more AI can do, the more we need to ask what it should and shouldn’t do. OpenAI researcher Jason Wolfe joins host Andrew Mayne to explore the Model Spec, the public framework that defines how models are intended to behave. They break down how it works in practice, from the chain of command that resolves conflicting instructions to the way it evolves over time through real-world use, feedback, and new model capabilities.

The more AI can do, the more we need to ask what it should and shouldn’t do. OpenAI researcher Jason Wolfe joins host Andrew Mayne to explore the Model Spec, the public framework that defines how models are intended to behave. They break down how it works in practice, from the chain of command that resolves conflicting instructions to the way it evolves over time through real-world use, feedback, and new model capabilities.

OpenAI

225,194 views • 3 months ago

We trained a robotic foundation model that can drive mobile robots in six different countries, and navigate Sproul Plaza in midday on the UC Berkeley campus! Some cool new work w/ noriaki_hirose, Lydia Ignatova, Kyle Stachowicz, Catherine Glossop, Dhruv Shah

We trained a robotic foundation model that can drive mobile robots in six different countries, and navigate Sproul Plaza in midday on the UC Berkeley campus! Some cool new work w/ noriaki_hirose, Lydia Ignatova, Kyle Stachowicz, Catherine Glossop, Dhruv Shah

Sergey Levine

52,979 views • 1 year ago

Some scene reconstruction R&D at Simulon that produces more detailed meshes quickly and is able to exclude moving objects without explicit segmentation. This improves occlusion of real-world objects and provides better meshes for 3D creators. Coming to the beta soon.

Some scene reconstruction R&D at Simulon that produces more detailed meshes quickly and is able to exclude moving objects without explicit segmentation. This improves occlusion of real-world objects and provides better meshes for 3D creators. Coming to the beta soon.

Divesh Naidoo

50,643 views • 2 years ago

Among the policies we evaluated so far (ACT, DiT, SmolVLA, pi0, pi0.5), fine-tuned pi0.5 achieves the best performance on VLA-REPLICA. The trend is consistent with recent simulation benchmarks such as RoboLab. The policy behaviors in the real world:

Among the policies we evaluated so far (ACT, DiT, SmolVLA, pi0, pi0.5), fine-tuned pi0.5 achieves the best performance on VLA-REPLICA. The trend is consistent with recent simulation benchmarks such as RoboLab. The policy behaviors in the real world:

Yu Xiang

19,066 views • 1 month ago

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐

📢 Announcing one of the most exciting works from us this year on scalable robot policy evaluation through real-to-sim transfer, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is orders of magnitude slower than the development of language agents. We propose a new framework where simulation performance strongly correlates with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing state-of-the-art correlation between simulation and reality for deformable object manipulation. It provides a scalable and reproducible evaluation engine for robot learning. 🌐

Yunzhu Li

39,900 views • 8 months ago

Years before AI infrastructure became the most competitive market in tech, we were proving that behind-the-meter computing works at scale. Bitcoin gave us that proof. Real assets. Real operations. Real performance through real market cycles. Now that same model — the same power strategy, the same operational discipline — is expanding to serve AI and high-performance compute demand. The workload evolves. The infrastructure holds. $SLNH

Years before AI infrastructure became the most competitive market in tech, we were proving that behind-the-meter computing works at scale. Bitcoin gave us that proof. Real assets. Real operations. Real performance through real market cycles. Now that same model — the same power strategy, the same operational discipline — is expanding to serve AI and high-performance compute demand. The workload evolves. The infrastructure holds. $SLNH

Soluna

11,115 views • 20 days ago

New course 🚨 Learn to build AI apps that can process very long documents with the Jamba model in this course, built in partnership with AI21 Labs and taught by Chen Wang and Chen Almagor. Learn more and join for free:

New course 🚨 Learn to build AI apps that can process very long documents with the Jamba model in this course, built in partnership with AI21 Labs and taught by Chen Wang and Chen Almagor. Learn more and join for free:

DeepLearning.AI

10,882 views • 1 year ago

“That was probably the lowest moment for me as a 12-year-old kid.” Andrew McCutchen can still remember the feeling of losing a championship game in Little League, but it shaped his ability to learn from and cope with failures on the field.

“That was probably the lowest moment for me as a 12-year-old kid.” Andrew McCutchen can still remember the feeling of losing a championship game in Little League, but it shaped his ability to learn from and cope with failures on the field.

MLB

466,523 views • 10 months ago

So you’ve trained your favorite diffusion/flow based policy, but it’s just not good enough 0-shot. Worry not, in our new work DSRL - we show how to *steer* pre-trained diffusion policies with off-policy RL, improving behavior efficiently enough for direct training in the real world! DSRL retains nice exploration from the base policy, but allows for quick improvement beyond this base policy with RL. The method is frustratingly simple, and super easy to throw on top of your favorite pretrained policy (VLA/diffusion policy, etc). Let’s think about how it works, 🧵 (1/10)

So you’ve trained your favorite diffusion/flow based policy, but it’s just not good enough 0-shot. Worry not, in our new work DSRL - we show how to steer pre-trained diffusion policies with off-policy RL, improving behavior efficiently enough for direct training in the real world! DSRL retains nice exploration from the base policy, but allows for quick improvement beyond this base policy with RL. The method is frustratingly simple, and super easy to throw on top of your favorite pretrained policy (VLA/diffusion policy, etc). Let’s think about how it works, 🧵 (1/10)

Abhishek Gupta

19,084 views • 1 year ago

I always dreamed of AGI as a wise advisor for humanity. Although LLMs are great for coding & knowledge work, I wouldn’t trust them to give me advice on my career, business strategy, or policy preferences. How can we build AI systems optimized for wisdom? At Mantic we believe the unlock is prediction: predicting world events as accurately as possible, and hill-climbing this single metric. Today we share some recent progress on the Thinking Machines website, having found Tinker a great platform for our RL experiments. TL;DR: We RL-tune gpt-oss-120b to become a better forecaster than any other model. Having good scaffolding is a prerequisite. A fun result: our tuned model + Grok are decorrelated from the other best models, and so are the most indispensable when picking a team.

I always dreamed of AGI as a wise advisor for humanity. Although LLMs are great for coding & knowledge work, I wouldn’t trust them to give me advice on my career, business strategy, or policy preferences. How can we build AI systems optimized for wisdom? At Mantic we believe the unlock is prediction: predicting world events as accurately as possible, and hill-climbing this single metric. Today we share some recent progress on the Thinking Machines website, having found Tinker a great platform for our RL experiments. TL;DR: We RL-tune gpt-oss-120b to become a better forecaster than any other model. Having good scaffolding is a prerequisite. A fun result: our tuned model + Grok are decorrelated from the other best models, and so are the most indispensable when picking a team.

Toby Shevlane

176,558 views • 3 months ago

We have a STRONG case to make, based on numbers, that the economy gets better right now — and can thrive in 2026. But that persuasion only works if we acknowledge the current pain & present reality! ⁦NewsNation⁩ ⁦Leland Vittert⁩

We have a STRONG case to make, based on numbers, that the economy gets better right now — and can thrive in 2026. But that persuasion only works if we acknowledge the current pain & present reality! ⁦NewsNation⁩ ⁦Leland Vittert⁩

Steve Cortes

238,443 views • 7 months ago