Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

A new paper by Disney Research introduces a 2-stage technique for motion tracking in physics-based character animation, using a pretrained latent space and RL to control diverse and unseen motions on both virtual and real-world humanoid robots. Paper:

TheHumanoidHub's profile picture

The Humanoid Hub

112,603 subscribers

20,088 Aufrufe • vor 2 Jahren •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

12 Kommentare

Profilbild von Rowan Cheung
Rowan Cheungvor 2 Jahren

This bot dances better than I do

Profilbild von The Humanoid Hub
The Humanoid Hubvor 2 Jahren

we're getting out-danced by our future overlords, and it's only 2024

Profilbild von PowerBeatsVR
PowerBeatsVRvor 3 Jahren

Get ready for a full-body VR workout that’s fun, fast, and intuitive — Play PowerBeatsVR (Now on Meta Quest) 🔥

Profilbild von SETI Park
SETI Parkvor 2 Jahren

Disney and ETH Zurich!

Profilbild von C Zhang
C Zhangvor 2 Jahren

there's no sim2real details in the paper. But these real-world results are far better than other related works except the Atlas dancing.

Profilbild von Oli
Olivor 2 Jahren

disney has an excellent robotics team i wouldnt be surprised if we see their animatronic characters walking around and interacting with people and the environment soon

Profilbild von Creative_Science💭
Creative_Science💭vor 2 Jahren

Another tasty research paper. 📜 Their unique way of blending animation and robotics, best I've seen in output so far.

Profilbild von Fabien Musty
Fabien Mustyvor 2 Jahren

They are firmly placed on top of a toy opportunity gold mine. Toys: gateway to the mainstream.

Profilbild von The Humanoid Hub
The Humanoid Hubvor 2 Jahren

💯 Toys -> Companion bots -> Helper bots.

Profilbild von Madman.Money
Madman.Moneyvor 2 Jahren

Disney is cooking everybody in robotics research. lol

Profilbild von Ilir Aliu - eu/acc
Ilir Aliu - eu/accvor 2 Jahren

So the robot is moving like this?

Profilbild von The Humanoid Hub
The Humanoid Hubvor 2 Jahren

@Ilir_AI The robot is executing the policies learned in simulation.

Ähnliche Videos

Another robot from Disney! 🕺🏻 Just look at this dancing fella created by Disney Research. Creating control policies that work on real robots and handling diverse, unseen motions in physics-based character control is still difficult. The Disney team proposes a two-state solution. The first step involves learning a latent space encoding with an autoencoder. After encoding, the policy is trained to map kinematic input to dynamic output, ensuring accurate and adaptable movement. This method eliminates common mode collapse issues and improves motion encoding by separating these stages. This technique has proven effective in real robots in simulations, marking a significant leap forward in robot control. Can't wait for it in action live in 2025 :) ~~~ RT to help 1 robot find a new workplace.
0:39

Another robot from Disney! 🕺🏻 Just look at this dancing fella created by Disney Research. Creating control policies that work on real robots and handling diverse, unseen motions in physics-based character control is still difficult. The Disney team proposes a two-state solution. The first step involves learning a latent space encoding with an autoencoder. After encoding, the policy is trained to map kinematic input to dynamic output, ensuring accurate and adaptable movement. This method eliminates common mode collapse issues and improves motion encoding by separating these stages. This technique has proven effective in real robots in simulations, marking a significant leap forward in robot control. Can't wait for it in action live in 2025 :) ~~~ RT to help 1 robot find a new workplace.

Lukas Ziegler

65,972 Aufrufe • vor 1 Jahr

RobotMDM, by Disney Research, combines diffusion-based motion generation with RL to produce physics-aware humanoid motions from text prompts. Trained on human motion data with a reward surrogate for physical feasibility, it ensures realistic motions.
0:56

RobotMDM, by Disney Research, combines diffusion-based motion generation with RL to produce physics-aware humanoid motions from text prompts. Trained on human motion data with a reward surrogate for physical feasibility, it ensures realistic motions.

The Humanoid Hub

22,943 Aufrufe • vor 1 Jahr

New research: Quadruped robots climbing ladders! 🦾🚀 Using RL-based control and a hooked end-effector, we're expanding robot capabilities in industrial environments. More info in the paper: Video:
0:45

New research: Quadruped robots climbing ladders! 🦾🚀 Using RL-based control and a hooked end-effector, we're expanding robot capabilities in industrial environments. More info in the paper: Video:

Robotic Systems Lab

16,390 Aufrufe • vor 1 Jahr

Meet Any2Track, a new two-stage RL framework designed to address a core challenge for humanoid robots: reliably tracking diverse, dynamic motions in the real world. The framework consists of two key components: AnyTracker, a general motion tracker, and AnyAdapter, a module that allows the robot to adapt to real-world disturbances like uneven terrain or external forces. This approach gives robots the stability needed for practical use. The system has been successfully deployed on a Unitree G1 humanoid robot with zero-shot sim2real transfer, performing exceptionally well in various motion tracking tasks under multiple real-world disturbances. Project: Codebase:
1:44

Meet Any2Track, a new two-stage RL framework designed to address a core challenge for humanoid robots: reliably tracking diverse, dynamic motions in the real world. The framework consists of two key components: AnyTracker, a general motion tracker, and AnyAdapter, a module that allows the robot to adapt to real-world disturbances like uneven terrain or external forces. This approach gives robots the stability needed for practical use. The system has been successfully deployed on a Unitree G1 humanoid robot with zero-shot sim2real transfer, performing exceptionally well in various motion tracking tasks under multiple real-world disturbances. Project: Codebase:

RoboHub🤖

61,378 Aufrufe • vor 11 Monaten

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.
1:05

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.

The Humanoid Hub

27,879 Aufrufe • vor 1 Jahr

Researchers from RAI Institute present Diffuse-CLoC, a new control policy that fuses kinematic motion diffusion models with physics-based control to produce motions that are both physically realistic and precisely controllable. This breakthrough moves us closer to developing generalist policies that enable humanoid robots to perform diverse tasks, including dynamic locomotion and contact-rich manipulation, in a natural-looking and robust way. Learn more at
0:31

Researchers from RAI Institute present Diffuse-CLoC, a new control policy that fuses kinematic motion diffusion models with physics-based control to produce motions that are both physically realistic and precisely controllable. This breakthrough moves us closer to developing generalist policies that enable humanoid robots to perform diverse tasks, including dynamic locomotion and contact-rich manipulation, in a natural-looking and robust way. Learn more at

RAI Institute

13,556 Aufrufe • vor 1 Jahr

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.
0:45

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.

The Humanoid Hub

20,466 Aufrufe • vor 1 Jahr

🤖 New paper: MobileVLA-R1 A unified VLA system that brings real reasoning + continuous control to quadruped robots. CoT dataset, 2-stage training, real-world deployment. 📄paper & code & demo:
3:48

🤖 New paper: MobileVLA-R1 A unified VLA system that brings real reasoning + continuous control to quadruped robots. CoT dataset, 2-stage training, real-world deployment. 📄paper & code & demo:

Hao Tang (hiring postdocs)

27,284 Aufrufe • vor 9 Monaten

Ego-VCP: a learned ego-vision world model trained offline on demonstration-free random data that predicts dynamics in latent space for humanoid robots. Achieved robust real-time contact-rich planning on a real Unitree G1: bracing against walls, blocking flying objects, traversing low arches. Paper:
1:14

Ego-VCP: a learned ego-vision world model trained offline on demonstration-free random data that predicts dynamics in latent space for humanoid robots. Achieved robust real-time contact-rich planning on a real Unitree G1: bracing against walls, blocking flying objects, traversing low arches. Paper:

The Humanoid Hub

24,742 Aufrufe • vor 9 Monaten

Congratulations to our NVIDIA Research team for SONIC being published in Science Magazine. 🦾 Built by NVIDIA Robotics researchers and collaborators, SONIC demonstrates scaling motion tracking for natural, robust humanoid whole-body control. Read the paper 📄
0:36

Congratulations to our NVIDIA Research team for SONIC being published in Science Magazine. 🦾 Built by NVIDIA Robotics researchers and collaborators, SONIC demonstrates scaling motion tracking for natural, robust humanoid whole-body control. Read the paper 📄

NVIDIA Robotics

51,626 Aufrufe • vor 23 Tagen

Let’s think about humanoid robots outside carrying the box. How about having the humanoid come out the door, interact with humans, and even dance? Introducing Expressive Whole-Body Control for Humanoid Robots: See how our robot performs rich, diverse, and expressive motions in the real world 👇🧵
2:37

Let’s think about humanoid robots outside carrying the box. How about having the humanoid come out the door, interact with humans, and even dance? Introducing Expressive Whole-Body Control for Humanoid Robots: See how our robot performs rich, diverse, and expressive motions in the real world 👇🧵

Xiaolong Wang

309,258 Aufrufe • vor 2 Jahren

Excited to share a new humanoid robot platform we’ve been working on. Berkeley Humanoid is a reliable and low-cost mid-scale research platform for learning-based control. We demonstrate the robot walking on various terrains and dynamic hopping with a simple RL controller.
0:49

Excited to share a new humanoid robot platform we’ve been working on. Berkeley Humanoid is a reliable and low-cost mid-scale research platform for learning-based control. We demonstrate the robot walking on various terrains and dynamic hopping with a simple RL controller.

Qiayuan Liao

80,436 Aufrufe • vor 2 Jahren

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.
1:11

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

The Humanoid Hub

25,125 Aufrufe • vor 9 Monaten

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 Aufrufe • vor 1 Jahr

Most motion papers tailor one controller to one specific task. This year at SIGGRAPH, our research team asks: can motor control itself be pretrained and reused? Generative Pretrained Controllers, or GPC, turn motor skills into a vocabulary of discrete tokens and train a transformer-based generative controller through next-token prediction. Just like GPT, the same pretrained controller can then be fine-tuned to solve new tasks. Trained on 600+ hours of motion, GPC runs in real-time inside a physics simulation, producing natural and physically grounded behaviors for interactive control.
1:45

Most motion papers tailor one controller to one specific task. This year at SIGGRAPH, our research team asks: can motor control itself be pretrained and reused? Generative Pretrained Controllers, or GPC, turn motor skills into a vocabulary of discrete tokens and train a transformer-based generative controller through next-token prediction. Just like GPT, the same pretrained controller can then be fine-tuned to solve new tasks. Trained on 600+ hours of motion, GPC runs in real-time inside a physics simulation, producing natural and physically grounded behaviors for interactive control.

NVIDIA AI

176,036 Aufrufe • vor 2 Monaten

"A Generative Motion Rig for Artist Driven Motion Authoring" from Disney Research Zurich and ETH Zurich So cool! Full video: Paper:
0:33

"A Generative Motion Rig for Artist Driven Motion Authoring" from Disney Research Zurich and ETH Zurich So cool! Full video: Paper:

Chris Offner

37,722 Aufrufe • vor 1 Monat

Olaf made a special appearance on stage with NVIDIA CEO Jensen Huang at #NVIDIAGTC.☃️ NVIDIA and Disney Research are working together to bring this beloved character into the real world using Newton, the open-source physics engine.
0:59

Olaf made a special appearance on stage with NVIDIA CEO Jensen Huang at #NVIDIAGTC.☃️ NVIDIA and Disney Research are working together to bring this beloved character into the real world using Newton, the open-source physics engine.

NVIDIA Robotics

630,008 Aufrufe • vor 5 Monaten

'LATENT' learns tennis skills for humanoid robots from human motion data. The robot can sustain multi-shot rallies, handle ball speeds of 15+ m/s, and showed a 90.9% success rate for the forehand. No onboard cameras or vision models, relies on external MoCap for high-precision, low-latency ball tracking. Paper:
1:31

'LATENT' learns tennis skills for humanoid robots from human motion data. The robot can sustain multi-shot rallies, handle ball speeds of 15+ m/s, and showed a 90.9% success rate for the forehand. No onboard cameras or vision models, relies on external MoCap for high-precision, low-latency ball tracking. Paper:

The Humanoid Hub

63,824 Aufrufe • vor 5 Monaten