Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🤖 Introducing Human-Object Interaction from Human-Level Instructions! First complete system that generates physically plausible, long-horizon human-object interactions with finger motions in contextual environments, driven by human-level instructions. 🔍 Our approach: - LLMs transform instructions into execution plans & target layouts - Multi-stage framework generates synchronized object, body & finger...

jiaman01's profile picture

Jiaman Li

1,295 subscribers

104,574 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieGesundheit & WellnessBildung
LIVE

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Michael Black
Michael Blackvor 1 Jahr

Great work!

Profilbild von Jiaman Li
Jiaman Livor 1 Jahr

Thank you!

Profilbild von AssemblyAI
AssemblyAIvor 1 Jahr

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

Profilbild von The Highly Automated Cat — e/acc ⏩
The Highly Automated Cat — e/acc ⏩vor 1 Jahr

in the future we will all be playing w doll houses

Profilbild von Johnny Núñez
Johnny Núñezvor 1 Jahr

omg it is really cool! I love it

Profilbild von Abhinav Girdhar
Abhinav Girdharvor 1 Jahr

Incredible advancement! Human-object interaction driven by human-level instructions is a huge leap forward. Excited to see how this multi-stage framework reshapes AI applications in realistic simulations!

Profilbild von satheesh kola
satheesh kolavor 1 Jahr

Great work!

Profilbild von satheesh kola
satheesh kolavor 1 Jahr

Loved it!

Profilbild von Patrick Kwon
Patrick Kwonvor 1 Jahr

Amazing!

Profilbild von Nicolas Keller
Nicolas Kellervor 1 Jahr

This looks amazing! :)

Profilbild von MichaelS
MichaelSvor 1 Jahr

Super cool👏👏👏

Ähnliche Videos

New study introduces a system to generate realistic, physically plausible human-object interactions from natural language instructions. It uses LLMs for planning, a motion generator for synchronized body and finger movements, and RL for physical accuracy.
1:22

New study introduces a system to generate realistic, physically plausible human-object interactions from natural language instructions. It uses LLMs for planning, a motion generator for synchronized body and finger movements, and RL for physical accuracy.

The Humanoid Hub

27,606 Aufrufe • vor 1 Jahr

Any2Track tracks diverse motions under various disturbances. The two-stage RL framework learns to track contact-rich human motions from motion capture datasets and adds adaptability to disturbances via dynamics-aware model prediction.
1:19

Any2Track tracks diverse motions under various disturbances. The two-stage RL framework learns to track contact-rich human motions from motion capture datasets and adds adaptability to disturbances via dynamics-aware model prediction.

The Humanoid Hub

144,422 Aufrufe • vor 8 Monaten

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉
0:20

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉

Angela Dai

10,311 Aufrufe • vor 1 Monat

🤖Simulation-Ready Human–Scene Interaction Reconstruction🤖 #HSImul3R reconstructs sim-ready 3D human-scene interactions from casual videos with *physics-in-the-loop* that can be directly deployed to humanoids. - Project: - Code:
0:40

🤖Simulation-Ready Human–Scene Interaction Reconstruction🤖 #HSImul3R reconstructs sim-ready 3D human-scene interactions from casual videos with *physics-in-the-loop* that can be directly deployed to humanoids. - Project: - Code:

Ziwei Liu

15,326 Aufrufe • vor 2 Monaten

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.
0:45

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 Aufrufe • vor 8 Monaten

Apple's AI researchers introduce EMOTION, a framework for expressive humanoid gestures. ⦿ LLMs interpret social context and generate motions from human demos ⦿ Robots execute motions via inverse kinematics ⦿ Gestures are refined through human feedback Hardware: Fourier GR-1
1:24

Apple's AI researchers introduce EMOTION, a framework for expressive humanoid gestures. ⦿ LLMs interpret social context and generate motions from human demos ⦿ Robots execute motions via inverse kinematics ⦿ Gestures are refined through human feedback Hardware: Fourier GR-1

The Humanoid Hub

66,831 Aufrufe • vor 1 Jahr

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.
1:08

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

The Humanoid Hub

20,358 Aufrufe • vor 4 Monaten

Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 🎥 1/9
0:54

Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL: - 5 rewards, - 4 DR terms, - Proprio. ONLY, - NO history/curriculum. Ready for agile, human-like 🤖? (Best with 🎧) 🔗 🎥 1/9

Zhen Wu

806,605 Aufrufe • vor 8 Monaten

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels
4:10

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

Xiao Ma

46,260 Aufrufe • vor 10 Monaten

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 Aufrufe • vor 1 Jahr

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:
3:13

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Siyuan Huang

21,892 Aufrufe • vor 7 Monaten

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.
0:43

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

120,135 Aufrufe • vor 1 Jahr

🔥 Introducing MVLift: Generate realistic 3D motion without any 3D training data - just using 2D poses from monocular videos! Applicable to human motion, human-object interaction & animal motion. Joint work w/ Jiajun Wu & Karen 💡 How? We reformulate 3D motion estimation as generating consistent multi-view 2D pose sequences. Our framework uses 2D motion diffusion to progressively establish multi-view consistency, requiring only single-view 2D pose sequences for training. Project: Video with demonstration: Paper:
0:21

🔥 Introducing MVLift: Generate realistic 3D motion without any 3D training data - just using 2D poses from monocular videos! Applicable to human motion, human-object interaction & animal motion. Joint work w/ Jiajun Wu & Karen 💡 How? We reformulate 3D motion estimation as generating consistent multi-view 2D pose sequences. Our framework uses 2D motion diffusion to progressively establish multi-view consistency, requiring only single-view 2D pose sequences for training. Project: Video with demonstration: Paper:

Jiaman Li

15,788 Aufrufe • vor 1 Jahr

Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box
1:35

Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box

AK

205,943 Aufrufe • vor 2 Jahren

Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️ It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵
0:23

Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️ It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵

Google DeepMind

1,061,476 Aufrufe • vor 2 Jahren

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:
0:25

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Ilir Aliu - eu/acc

61,491 Aufrufe • vor 8 Monaten

New video of Clone Torso, demonstrating biomimicry of complex, natural motions of human shoulders.
1:05

New video of Clone Torso, demonstrating biomimicry of complex, natural motions of human shoulders.

The Humanoid Hub

494,970 Aufrufe • vor 1 Jahr