Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Train your avatars to interact with 3D scenes. We use adversarial imitation learning and reinforcement learning to train physically-simulated characters that perform scene interaction tasks in a natural and life-like manner. Today at #SIGGRAPH2023.

Michael Black

99,718 subscribers

46,237 views • 3 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

7 Comments

Michael Black3 years ago

We learn scene interaction behaviors from large unstructured motion datasets using an adversarial discriminator, without manual annotation of the mocap. The key novelty involves conditioning both the discriminator and the policy networks on scene context.

Michael Black3 years ago

Our policies learn to seamlessly transition between different behaviors like idling, walking, and sitting. By randomizing the properties of the objects and their placements during training, our method generalizes beyond the objects and scenarios depicted in the training data.

Michael Black3 years ago

Great @MPI_IS and @nvidia collaboration with @M_E_Hassan, Yunrong Guo, @TingwuWang, @FidlerSanja, @xbpeng4. PDF: Project: Video:

Tom Goldstein3 years ago

Hate to be picky but pretty sure that last yellow block to the groin is not handled in a “life-like” manner.

pieter ter doest3 years ago

wow

Mickel Liu3 years ago

This might just be the cruelest humanoid demo that I have ever seen 😂 jk

AllThingsAI3 years ago

This could easily be the most adorable humanoid demonstration I've come across.

Related Videos

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,813 views • 7 months ago

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,772 views • 2 months ago

Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. 1/

Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. 1/

Ajay Mandlekar

35,683 views • 1 year ago

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)

naklecha

46,795 views • 1 year ago

Learning about active perception with Haoyu Xiong -- your robot needs a head, and to be able to control where it's looking, in order to perform complex tasks!

Learning about active perception with Haoyu Xiong -- your robot needs a head, and to be able to control where it's looking, in order to perform complex tasks!

Chris Paxton

22,784 views • 11 months ago

Learning Physically Simulated Tennis Skills from Broadcast Videos demonstrate that system produces controllers for physically-simulated tennis players that can hit the incoming ball to target positions accurately using a diverse array of strokes (serves, forehands, and backhands), spins (topspins and slices), and playing styles (one/two-handed backhands, left/right-handed play). Overall, system can synthesize two physically simulated characters playing extended tennis rallies with simulated racket and ball dynamics paper: project page:

Learning Physically Simulated Tennis Skills from Broadcast Videos demonstrate that system produces controllers for physically-simulated tennis players that can hit the incoming ball to target positions accurately using a diverse array of strokes (serves, forehands, and backhands), spins (topspins and slices), and playing styles (one/two-handed backhands, left/right-handed play). Overall, system can synthesize two physically simulated characters playing extended tennis rallies with simulated racket and ball dynamics paper: project page:

AK

125,502 views • 3 years ago

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Felix Heide

27,779 views • 1 year ago

Hey Devs, ready to level up your AI skills? Build, train, and deploy with Vertex AI using our hands-on learning and expert-led content—designed to help you create smarter, scalable AI solutions. 🔗 Sign up and start learning today:

Hey Devs, ready to level up your AI skills? Build, train, and deploy with Vertex AI using our hands-on learning and expert-led content—designed to help you create smarter, scalable AI solutions. 🔗 Sign up and start learning today:

Google Cloud India

2,007,454 views • 1 year ago

It was great to see our name amongst the other “AI Native” companies during @Nvidia’s #GTC keynote. NVIDIA Isaac™ Lab helps us train reinforcement learning policies that enable the UMV to drive, jump, flip, and hop like a pro!

It was great to see our name amongst the other “AI Native” companies during @Nvidia’s #GTC keynote. NVIDIA Isaac™ Lab helps us train reinforcement learning policies that enable the UMV to drive, jump, flip, and hop like a pro!

RAI Institute

611,225 views • 5 months ago

Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned:

Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned:

Skyfall AI

268,264 views • 1 month ago

How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇

How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇

Chen Wang

288,716 views • 3 years ago

"Toward General Virtual Agents" I recently gave a talk at MIT. I argued that we should use tools from reinforcement learning and search to improve the capability and alignment of LLM agents. Slides: Video:

"Toward General Virtual Agents" I recently gave a talk at MIT. I argued that we should use tools from reinforcement learning and search to improve the capability and alignment of LLM agents. Slides: Video:

Stephen McAleer

167,206 views • 2 years ago

Frontier models that use reasoning to "think" during inference are generating 5X more AI tokens per year. ✨ "Inference is now a thinking process. And in order to teach AI how to think, reinforcement learning and very significant computation was introduced into post-training," said NVIDIA CEO Jensen Huang in a recent keynote at #CES2026. Reinforcement learning is increasing computation demands across all AI scaling laws: pre-training, post-training, and test-time scaling. Learn more about reinforcement learning and AI scaling ➡️

Frontier models that use reasoning to "think" during inference are generating 5X more AI tokens per year. ✨ "Inference is now a thinking process. And in order to teach AI how to think, reinforcement learning and very significant computation was introduced into post-training," said NVIDIA CEO Jensen Huang in a recent keynote at #CES2026. Reinforcement learning is increasing computation demands across all AI scaling laws: pre-training, post-training, and test-time scaling. Learn more about reinforcement learning and AI scaling ➡️

NVIDIA AI Infrastructure

13,329 views • 7 months ago

Last semester, I taught Reinforcement Learning class again at UCLA. Together with my amazing TAs Matthew and Caiyuan, we built a mini-project: MetaDrive Arena 🚗🤖 Students applied what they learned in class, trained RL agents, and competed on a live leaderboard. The results were incredible, with 94 agents, 2K submissions, and 130K matches. We saw tons of creativity, clever ideas, and real progress in learning. We’re now releasing it publicly to support RL education and experimentation. Try it out and train your own agent at 🔗

Last semester, I taught Reinforcement Learning class again at UCLA. Together with my amazing TAs Matthew and Caiyuan, we built a mini-project: MetaDrive Arena 🚗🤖 Students applied what they learned in class, trained RL agents, and competed on a live leaderboard. The results were incredible, with 94 agents, 2K submissions, and 130K matches. We saw tons of creativity, clever ideas, and real progress in learning. We’re now releasing it publicly to support RL education and experimentation. Try it out and train your own agent at 🔗

Bolei Zhou

25,253 views • 5 months ago

To bring this Olaf to life, Disney used A.I. called "reinforcement learning" to achieve self-walking and movements with creative intent. The figure has an articulated mouth, eyes and removable carrot nose & arms, and can interact and speak with guests. #OlafRobot #WorldofFrozen

To bring this Olaf to life, Disney used A.I. called "reinforcement learning" to achieve self-walking and movements with creative intent. The figure has an articulated mouth, eyes and removable carrot nose & arms, and can interact and speak with guests. #OlafRobot #WorldofFrozen

DSNY Newscast

82,366 views • 9 months ago

Very excited to announce: Keypoint Action Tokens! We found that LLMs can be repurposed as "imitation learning engines" for robots, by representing both observations & actions as 3D keypoints, and feeding into an LLM for in-context learning. See: More 👇

Very excited to announce: Keypoint Action Tokens! We found that LLMs can be repurposed as "imitation learning engines" for robots, by representing both observations & actions as 3D keypoints, and feeding into an LLM for in-context learning. See: More 👇

Edward Johns

32,569 views • 2 years ago

⚡️BREAKING: micro:bit CreateAI launches today 🎉 ! This World Children’s Day, use micro:bit CreateAI to inspire students to play a positive role in shaping the future of AI. Use your movement data and micro:bit to train a machine learning model - and then use it in code. 🤩 We're so excited - read more! #WorldChildrensDay #microbit #createAI

⚡️BREAKING: micro:bit CreateAI launches today 🎉 ! This World Children’s Day, use micro:bit CreateAI to inspire students to play a positive role in shaping the future of AI. Use your movement data and micro:bit to train a machine learning model - and then use it in code. 🤩 We're so excited - read more! #WorldChildrensDay #microbit #createAI

Micro:bit Educational Foundation

12,297 views • 1 year ago

R+X was accepted at ICRA 2025! Robots can now do in-context imitation learning, just by observing humans going about their daily lives... No more need to *label* and *train* - just *RETRIEVE* and *EXECUTE*! 🧵👇 (1/5)

R+X was accepted at ICRA 2025! Robots can now do in-context imitation learning, just by observing humans going about their daily lives... No more need to label and train - just RETRIEVE and EXECUTE! 🧵👇 (1/5)

Edward Johns

10,402 views • 1 year ago