Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Ever wish a robot could just move to any goal in any environment—avoiding all collisions and reacting in real time? 🚀Excited to share our #CoRL2025 paper, Deep Reactive Policy (DRP), a learning-based motion planner that navigates complex scenes with moving obstacles—directly from point cloud input. w/ Jiahui(Jim) Yang (1/N)

Jason Liu

2,104 subscribers

72,301 views • 11 months ago •via X (Twitter)

Science & Technology Education #CoRL2025

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Motion planning in complex tasks is hard and still done via slow, explicit, traditional planners. We present a generalist Neural Motion Planner -- a single neural network that plans complex dynamic motions quickly and accurately at test time. Building upon our lab's sim2real efforts, the key idea is to create many complex scenes in simulation and then distill classical motion planner trajectories into a single reactive neural network policy. More details in the thread below! 👇 Open-sourced:

Motion planning in complex tasks is hard and still done via slow, explicit, traditional planners. We present a generalist Neural Motion Planner -- a single neural network that plans complex dynamic motions quickly and accurately at test time. Building upon our lab's sim2real efforts, the key idea is to create many complex scenes in simulation and then distill classical motion planner trajectories into a single reactive neural network policy. More details in the thread below! 👇 Open-sourced:

Deepak Pathak

28,642 views • 1 year ago

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 views • 1 year ago

Introducing Open-TeleVision: with Fully Autonomous policy video👇. We can conduct a long-horizon task with inserting 12 cans nonstop without any interruptions. We offer: 🤖 Highly precise and smooth bimanual manipulation. 📺 Active egocentric vision (with a moving neck) feedback. It is achieved by imitation learning from teleoperation: We propose a VR-based REAL-TIME teleoperation that streams the stereo video observation from the robot camera to the VR device. The robot neck moves as the human head moves, the robot hands move as the human hands move, offering the operator an intuitive experience as the human herself becomes the robot. The devils are all in the details, and how to implement things right: ✅How to perform IK/retargeting for smooth and precise control. ✅How to do all these and also stream stereo video without no latency, all in real time. We released our code here: Active head hardware design: 1/n

Introducing Open-TeleVision: with Fully Autonomous policy video👇. We can conduct a long-horizon task with inserting 12 cans nonstop without any interruptions. We offer: 🤖 Highly precise and smooth bimanual manipulation. 📺 Active egocentric vision (with a moving neck) feedback. It is achieved by imitation learning from teleoperation: We propose a VR-based REAL-TIME teleoperation that streams the stereo video observation from the robot camera to the VR device. The robot neck moves as the human head moves, the robot hands move as the human hands move, offering the operator an intuitive experience as the human herself becomes the robot. The devils are all in the details, and how to implement things right: ✅How to perform IK/retargeting for smooth and precise control. ✅How to do all these and also stream stereo video without no latency, all in real time. We released our code here: Active head hardware design: 1/n

Xiaolong Wang

25,572 views • 2 years ago

Excited to share our #NeurIPS2025 work on learning motion hierarchies! We introduce a general hierarchical graph learning method that learns structured, interpretable motion directly from data, no prior structure or assumptions needed!!! Project and Paper: Amazing work led by William Koch, Cheng Zheng, and Baiang Li ! See us in San Diego for #NeurIPS2025!

Excited to share our #NeurIPS2025 work on learning motion hierarchies! We introduce a general hierarchical graph learning method that learns structured, interpretable motion directly from data, no prior structure or assumptions needed!!! Project and Paper: Amazing work led by William Koch, Cheng Zheng, and Baiang Li ! See us in San Diego for #NeurIPS2025!

Felix Heide

25,314 views • 8 months ago

1/ 🧠Humans are the best robot data source! 2/ 👓Human egocentric video is rich in quantity, but poor in quality. 3/ Beyond scaling data, smarter representation and architecture matter just as much. 4/ Want an open-source framework to train your own learn-from-human-data robot policy? 🚀We introduce HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Robot-Data-Free ✦ Just 30 min of data per task ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Easy-to-Implement Let's squeeze every bit of signal out of human data! 🌐 Website: 📄 Paper: 💻 Code: 📹 Video: 🧵 1/n

1/ 🧠Humans are the best robot data source! 2/ 👓Human egocentric video is rich in quantity, but poor in quality. 3/ Beyond scaling data, smarter representation and architecture matter just as much. 4/ Want an open-source framework to train your own learn-from-human-data robot policy? 🚀We introduce HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Robot-Data-Free ✦ Just 30 min of data per task ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Easy-to-Implement Let's squeeze every bit of signal out of human data! 🌐 Website: 📄 Paper: 💻 Code: 📹 Video: 🧵 1/n

Zhi (Leo) Wang

110,230 views • 2 months ago

We’re excited to share LuxRemix: interactive light editing for indoor scenes! 🏠💡 Capture a room once, then turn individual lights on/off, change colors, and adjust intensity – all in real-time 3D from any viewpoint. 💡 📄

We’re excited to share LuxRemix: interactive light editing for indoor scenes! 🏠💡 Capture a room once, then turn individual lights on/off, change colors, and adjust intensity – all in real-time 3D from any viewpoint. 💡 📄

Christian Richardt

10,718 views • 6 months ago

Legged Locomotion… meets Skateboarding [Paper ⬇️] Most robot movement models either rely on fixed patterns or struggle to handle complex changes. DHAL (Discrete-time Hybrid Automata Learning) takes a different approach: using reinforcement learning to teach robots when and how to switch movements in real-time: ✅ Learns when to switch between different motions without pre-labeled data ✅ Handles complex, high-dimensional movements like a quadrupedal robot on a skateboard ✅ Uses a multi-critic architecture to improve contact-based motion control ✅ Works in both simulation and real-world environments with strong results It proves that robots can learn movement transitions on their own, without predefined rules. Paper: Thanks to Hang Liu for bringing this to my attention!

Legged Locomotion… meets Skateboarding [Paper ⬇️] Most robot movement models either rely on fixed patterns or struggle to handle complex changes. DHAL (Discrete-time Hybrid Automata Learning) takes a different approach: using reinforcement learning to teach robots when and how to switch movements in real-time: ✅ Learns when to switch between different motions without pre-labeled data ✅ Handles complex, high-dimensional movements like a quadrupedal robot on a skateboard ✅ Uses a multi-critic architecture to improve contact-based motion control ✅ Works in both simulation and real-world environments with strong results It proves that robots can learn movement transitions on their own, without predefined rules. Paper: Thanks to Hang Liu for bringing this to my attention!

Ilir Aliu - eu/acc

41,894 views • 1 year ago

Our third announcement is GWM Worlds. A world model for real-time environment simulation. You give the model a static scene, and it generates an immersive, infinite, explorable space as you move through it, with geometry, lighting and physics. All in real-time. You can travel to any place, real or imagined. You can become any agent, a person walking through a city, a drone flying over a snowy mountain or a robot navigating a warehouse.

Our third announcement is GWM Worlds. A world model for real-time environment simulation. You give the model a static scene, and it generates an immersive, infinite, explorable space as you move through it, with geometry, lighting and physics. All in real-time. You can travel to any place, real or imagined. You can become any agent, a person walking through a city, a drone flying over a snowy mountain or a robot navigating a warehouse.

Runway

80,385 views • 8 months ago

🎉Excited to share a fun little hardware project we’ve been working on. GELLO is an intuitive and low cost teleoperation device for robot arms that costs less than $300. We've seen the importance of data quality in imitation learning. Our goal is to make this more accessible 1/n

🎉Excited to share a fun little hardware project we’ve been working on. GELLO is an intuitive and low cost teleoperation device for robot arms that costs less than $300. We've seen the importance of data quality in imitation learning. Our goal is to make this more accessible 1/n

Philipp Wu

162,934 views • 2 years ago

A system with moving transfer plates! 🍽️ Modular transfer plates that move items in any direction. These systems from Festo give factories something really cool; reconfigurable motion that adapts as layouts or processes change. Because they can route items in multiple directions, you can run inspections, branch flows based on sensor data, or perform operations directly on the moving parts. It could even power fully automated storage: shelves that reposition items based on real-time demand. I'd love to see more of them in 3PL centers. Where would you use them? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

A system with moving transfer plates! 🍽️ Modular transfer plates that move items in any direction. These systems from Festo give factories something really cool; reconfigurable motion that adapts as layouts or processes change. Because they can route items in multiple directions, you can run inspections, branch flows based on sensor data, or perform operations directly on the moving parts. It could even power fully automated storage: shelves that reposition items based on real-time demand. I'd love to see more of them in 3PL centers. Where would you use them? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

214,305 views • 15 days ago

🤖✨Excited to share our new work: OMG: Omni-Modal Motion Generation for Generalist Humanoid Control What if a humanoid could understand intent from language, music/audio, human motion, or their combinations—and turn it into executable whole-body motion in real time? [🧵1/11]

🤖✨Excited to share our new work: OMG: Omni-Modal Motion Generation for Generalist Humanoid Control What if a humanoid could understand intent from language, music/audio, human motion, or their combinations—and turn it into executable whole-body motion in real time? [🧵1/11]

Siqiao Huang

13,261 views • 2 months ago

A system with moving transfer plates! 🍽️ Modular transfer plates that move items in any direction! These systems from Festo give factories something traditional conveyors can’t: fine, reconfigurable motion that adapts as layouts or processes change. Because they can route items in multiple directions, you can run inspections, branch flows based on sensor data, or perform operations directly on the moving parts. It could even power fully automated storage: shelves that reposition items based on real-time demand. I'd love to see more of them in 3PL centers. Where would you use them? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

A system with moving transfer plates! 🍽️ Modular transfer plates that move items in any direction! These systems from Festo give factories something traditional conveyors can’t: fine, reconfigurable motion that adapts as layouts or processes change. Because they can route items in multiple directions, you can run inspections, branch flows based on sensor data, or perform operations directly on the moving parts. It could even power fully automated storage: shelves that reposition items based on real-time demand. I'd love to see more of them in 3PL centers. Where would you use them? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

327,650 views • 8 months ago

Another robot from Disney! 🕺🏻 Just look at this dancing fella created by Disney Research. Creating control policies that work on real robots and handling diverse, unseen motions in physics-based character control is still difficult. The Disney team proposes a two-state solution. The first step involves learning a latent space encoding with an autoencoder. After encoding, the policy is trained to map kinematic input to dynamic output, ensuring accurate and adaptable movement. This method eliminates common mode collapse issues and improves motion encoding by separating these stages. This technique has proven effective in real robots in simulations, marking a significant leap forward in robot control. Can't wait for it in action live in 2025 :) ~~~ RT to help 1 robot find a new workplace.

Another robot from Disney! 🕺🏻 Just look at this dancing fella created by Disney Research. Creating control policies that work on real robots and handling diverse, unseen motions in physics-based character control is still difficult. The Disney team proposes a two-state solution. The first step involves learning a latent space encoding with an autoencoder. After encoding, the policy is trained to map kinematic input to dynamic output, ensuring accurate and adaptable movement. This method eliminates common mode collapse issues and improves motion encoding by separating these stages. This technique has proven effective in real robots in simulations, marking a significant leap forward in robot control. Can't wait for it in action live in 2025 :) ~~~ RT to help 1 robot find a new workplace.

Lukas Ziegler

65,955 views • 1 year ago

Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing discuss: We present a novel approach to synthesize dexterous motions for physically simulated hands in tasks that require coordination between the control of two hands with high temporal precision. Instead of directly learning a joint policy to control two hands, our approach performs bimanual control through cooperative learning where each hand is treated as an individual agent. The individual policies for each hand are first trained separately, and then synchronized through latent space manipulation in a centralized environment to serve as a joint policy for two-hand control. By doing so, we avoid directly performing policy learning in the joint state-action space of two hands with higher dimensions, greatly improving the overall training efficiency. We demonstrate the effectiveness of our proposed approach in the challenging guitar-playing task. The virtual guitarist trained by our approach can synthesize motions from unstructured reference data of general guitar-playing practice motions, and accurately play diverse rhythms with complex chord pressing and string picking patterns based on the input guitar tabs that do not exist in the references. Along with this paper, we provide the motion capture data that we collected as the reference for policy training.

Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing discuss: We present a novel approach to synthesize dexterous motions for physically simulated hands in tasks that require coordination between the control of two hands with high temporal precision. Instead of directly learning a joint policy to control two hands, our approach performs bimanual control through cooperative learning where each hand is treated as an individual agent. The individual policies for each hand are first trained separately, and then synchronized through latent space manipulation in a centralized environment to serve as a joint policy for two-hand control. By doing so, we avoid directly performing policy learning in the joint state-action space of two hands with higher dimensions, greatly improving the overall training efficiency. We demonstrate the effectiveness of our proposed approach in the challenging guitar-playing task. The virtual guitarist trained by our approach can synthesize motions from unstructured reference data of general guitar-playing practice motions, and accurately play diverse rhythms with complex chord pressing and string picking patterns based on the input guitar tabs that do not exist in the references. Along with this paper, we provide the motion capture data that we collected as the reference for policy training.

AK

26,855 views • 1 year ago

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

Haoran Geng

95,549 views • 1 year ago

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Abhishek Gupta

10,803 views • 1 year ago

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,080 views • 5 months ago

We’re excited to share our groundbreaking new audio-driven performance model, featuring hyper-real expressions in near real-time. Any length video, in any style, is ready in 6 seconds or less—in HD. And we’ve managed to make it 20x faster and cheaper 💅 It’s all part of our vision to make AI video accessible and fun for more people—a way for them to connect, not just create. Rolling out soon in the Pika social app.

We’re excited to share our groundbreaking new audio-driven performance model, featuring hyper-real expressions in near real-time. Any length video, in any style, is ready in 6 seconds or less—in HD. And we’ve managed to make it 20x faster and cheaper 💅 It’s all part of our vision to make AI video accessible and fun for more people—a way for them to connect, not just create. Rolling out soon in the Pika social app.

Pika

1,243,320 views • 1 year ago

Enabling autonomous vehicles perceive their environment using only off-the-shelf cameras has been a long term research objective at Swaayatt Robots. This demo highlights the capabilities of our on-road perception system which is able to detect obstacles, road boundaries, lane markers in images, as well as compute depth of the complex scenes in the environment. The output shown in this video is end-to-end raw output from our deep learning system, without any post processing. The current system, with joint computation of obstacles, lane/road boundaries, and depth, works at 30 FPS on an embedded GPU in our autonomous vehicle, and can achieve higher FPS with further optimization -- which is currently a research in progress. This system is being scaled up for both the day and night operations, and we will showcase its strength towards enabling autonomous driving on a mountainous environment with unpaved roads, in the absence of any delimiters. #deeplearning #autonomousdriving #autonomousvehicles #machinelearning

Enabling autonomous vehicles perceive their environment using only off-the-shelf cameras has been a long term research objective at Swaayatt Robots. This demo highlights the capabilities of our on-road perception system which is able to detect obstacles, road boundaries, lane markers in images, as well as compute depth of the complex scenes in the environment. The output shown in this video is end-to-end raw output from our deep learning system, without any post processing. The current system, with joint computation of obstacles, lane/road boundaries, and depth, works at 30 FPS on an embedded GPU in our autonomous vehicle, and can achieve higher FPS with further optimization -- which is currently a research in progress. This system is being scaled up for both the day and night operations, and we will showcase its strength towards enabling autonomous driving on a mountainous environment with unpaved roads, in the absence of any delimiters. #deeplearning #autonomousdriving #autonomousvehicles #machinelearning

Sanjeev Sharma

18,478 views • 1 year ago