Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🤖 Introducing Human-Object Interaction from Human-Level Instructions! First complete system that generates physically plausible, long-horizon human-object interactions with finger motions in contextual environments, driven by human-level instructions. 🔍 Our approach: - LLMs transform instructions into execution plans & target layouts - Multi-stage framework generates synchronized object, body & finger... show more

Jiaman Li

1,295 subscribers

104,574 views • 1 year ago •via X (Twitter)

Science & Technology Health & Wellness Education

Anya Rossi• Live Now

Private livecam show

11 Comments

Michael Black1 year ago

Great work!

Jiaman Li1 year ago

Thank you!

AssemblyAI1 year ago

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

The Highly Automated Cat — e/acc ⏩1 year ago

in the future we will all be playing w doll houses

Johnny Núñez1 year ago

omg it is really cool! I love it

Abhinav Girdhar1 year ago

Incredible advancement! Human-object interaction driven by human-level instructions is a huge leap forward. Excited to see how this multi-stage framework reshapes AI applications in realistic simulations!

satheesh kola1 year ago

Great work!

satheesh kola1 year ago

Loved it!

Patrick Kwon1 year ago

Amazing!

Nicolas Keller1 year ago

This looks amazing! :)

MichaelS1 year ago

Super cool👏👏👏

Related Videos

New study introduces a system to generate realistic, physically plausible human-object interactions from natural language instructions. It uses LLMs for planning, a motion generator for synchronized body and finger movements, and RL for physical accuracy.

New study introduces a system to generate realistic, physically plausible human-object interactions from natural language instructions. It uses LLMs for planning, a motion generator for synchronized body and finger movements, and RL for physical accuracy.

The Humanoid Hub

27,606 views • 1 year ago

Any2Track tracks diverse motions under various disturbances. The two-stage RL framework learns to track contact-rich human motions from motion capture datasets and adds adaptability to disturbances via dynamics-aware model prediction.

Any2Track tracks diverse motions under various disturbances. The two-stage RL framework learns to track contact-rich human motions from motion capture datasets and adds adaptability to disturbances via dynamics-aware model prediction.

The Humanoid Hub

144,422 views • 8 months ago

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉

Angela Dai

10,350 views • 1 month ago

🤖Simulation-Ready Human–Scene Interaction Reconstruction🤖 #HSImul3R reconstructs sim-ready 3D human-scene interactions from casual videos with *physics-in-the-loop* that can be directly deployed to humanoids. - Project: - Code:

🤖Simulation-Ready Human–Scene Interaction Reconstruction🤖 #HSImul3R reconstructs sim-ready 3D human-scene interactions from casual videos with physics-in-the-loop that can be directly deployed to humanoids. - Project: - Code:

Ziwei Liu

15,326 views • 2 months ago

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 views • 8 months ago

Apple's AI researchers introduce EMOTION, a framework for expressive humanoid gestures. ⦿ LLMs interpret social context and generate motions from human demos ⦿ Robots execute motions via inverse kinematics ⦿ Gestures are refined through human feedback Hardware: Fourier GR-1

Apple's AI researchers introduce EMOTION, a framework for expressive humanoid gestures. ⦿ LLMs interpret social context and generate motions from human demos ⦿ Robots execute motions via inverse kinematics ⦿ Gestures are refined through human feedback Hardware: Fourier GR-1

The Humanoid Hub

66,831 views • 1 year ago

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

The Humanoid Hub

20,358 views • 4 months ago

Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 🎥 1/9

Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 🎥 1/9

Zhen Wu

807,937 views • 8 months ago

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

Xiao Ma

46,260 views • 10 months ago

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Siyuan Huang

21,892 views • 7 months ago

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

120,135 views • 1 year ago

🔥 Introducing MVLift: Generate realistic 3D motion without any 3D training data - just using 2D poses from monocular videos! Applicable to human motion, human-object interaction & animal motion. Joint work w/ Jiajun Wu & Karen 💡 How? We reformulate 3D motion estimation as generating consistent multi-view 2D pose sequences. Our framework uses 2D motion diffusion to progressively establish multi-view consistency, requiring only single-view 2D pose sequences for training. Project: Video with demonstration: Paper:

🔥 Introducing MVLift: Generate realistic 3D motion without any 3D training data - just using 2D poses from monocular videos! Applicable to human motion, human-object interaction & animal motion. Joint work w/ Jiajun Wu & Karen 💡 How? We reformulate 3D motion estimation as generating consistent multi-view 2D pose sequences. Our framework uses 2D motion diffusion to progressively establish multi-view consistency, requiring only single-view 2D pose sequences for training. Project: Video with demonstration: Paper:

Jiaman Li

15,788 views • 1 year ago

Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box

Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box

AK

205,943 views • 2 years ago

Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️ It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵

Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️ It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵

Google DeepMind

1,061,476 views • 2 years ago

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Ilir Aliu - eu/acc

61,491 views • 8 months ago

New video of Clone Torso, demonstrating biomimicry of complex, natural motions of human shoulders.

New video of Clone Torso, demonstrating biomimicry of complex, natural motions of human shoulders.

The Humanoid Hub

494,970 views • 1 year ago