Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introduce EgoInfinity: a web-scale (14.6 yrs, 142M clips) data engine that automatically lifts Youtube videos into 4D hand-object interaction, and retargets for robot learning Not only a dataset, but a modular, upgradable engine HF Space:

Vector Wang

2,190 subscribers

106,100 views • 6 days ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 views • 9 months ago

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

mirrash.eth 🦇🔊

250,988 views • 2 days ago

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

Julen Urain

42,434 views • 1 year ago

AGIBOT founder Zhihui Peng unveiled the Lingxi X2, a bipedal humanoid robot integrating motion, interaction, and task intelligence. With 28 degrees of freedom, a Diffusion-based motion engine, and a computing system that enables natural human-robot interaction. It can walk, run, dance, and ride a bicycle, while also excelling in precise object manipulation.

AGIBOT founder Zhihui Peng unveiled the Lingxi X2, a bipedal humanoid robot integrating motion, interaction, and task intelligence. With 28 degrees of freedom, a Diffusion-based motion engine, and a computing system that enables natural human-robot interaction. It can walk, run, dance, and ride a bicycle, while also excelling in precise object manipulation.

The Humanoid Hub

124,533 views • 1 year ago

Sikorsky H-34 unusual engine seen here in a video by AviationWalkarounds on YouTube. Unusual not only for its forward placement on an Helicopter, but also for being a radial engine.

Sikorsky H-34 unusual engine seen here in a video by AviationWalkarounds on YouTube. Unusual not only for its forward placement on an Helicopter, but also for being a radial engine.

Air Safety #OTD by Francisco Cunha

21,491 views • 4 months ago

Earlier this year, AgiBot open-sourced a large-scale dataset of real-world humanoid robot tasks. Now, they've announced AgiBot Digital World, a large-scale simulation framework that includes an open-sourced dataset covering 180+ object categories and 12 core skills.

Earlier this year, AgiBot open-sourced a large-scale dataset of real-world humanoid robot tasks. Now, they've announced AgiBot Digital World, a large-scale simulation framework that includes an open-sourced dataset covering 180+ object categories and 12 core skills.

The Humanoid Hub

43,709 views • 1 year ago

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without *any* robot teleop data collection 1/n

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without any robot teleop data collection 1/n

Homanga Bharadhwaj @ CVPR

28,408 views • 1 year ago

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 views • 1 year ago

Toyota IMV Origin is not just a pickup truck, but a modular work platform that the company intentionally makes "unfinished." Toyota supplies only the base — the body, chassis, and engine. Everything else is assembled by the owners.

Toyota IMV Origin is not just a pickup truck, but a modular work platform that the company intentionally makes "unfinished." Toyota supplies only the base — the body, chassis, and engine. Everything else is assembled by the owners.

Massimo

193,722 views • 7 months ago

We can convert human videos to robot hand-object interaction trajectories in 4D. Enjoy! Paper: Website: Code: Authors:Bhawna Paliwal,Haritheja,Will Liang, Pieter Abbeel , Mahi Shafiullah 🏠🤖 , Jitendra MALIK

We can convert human videos to robot hand-object interaction trajectories in 4D. Enjoy! Paper: Website: Code: Authors:Bhawna Paliwal,Haritheja,Will Liang, Pieter Abbeel , Mahi Shafiullah 🏠🤖 , Jitendra MALIK

Jitendra MALIK

57,457 views • 6 days ago

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

The Humanoid Hub

20,358 views • 4 months ago

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

MrNeRF

42,534 views • 10 months ago

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,228 views • 1 year ago

Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior. We’re releasing GRAIL: — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)

Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior. We’re releasing GRAIL: — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)

Ye Yuan

42,280 views • 20 days ago

A hydraulic lifter is a small but powerful component that plays a huge role in keeping your engine smooth, quiet, and efficient. It sits between the camshaft and the engine's valves, automatically maintaining the perfect valve clearance while the engine runs.

A hydraulic lifter is a small but powerful component that plays a huge role in keeping your engine smooth, quiet, and efficient. It sits between the camshaft and the engine's valves, automatically maintaining the perfect valve clearance while the engine runs.

Mechanical Knowledge

11,373 views • 4 months ago

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Felix Heide

27,763 views • 9 months ago

Want a robot that learns household tasks by watching you? EquiBot is a ✨ generalizable and 🚰 data-efficient method for visuomotor policy learning, robust to changes in object shapes, lighting, and scene makeup, even from just 5 mins of human videos. 🧵↓

Want a robot that learns household tasks by watching you? EquiBot is a ✨ generalizable and 🚰 data-efficient method for visuomotor policy learning, robust to changes in object shapes, lighting, and scene makeup, even from just 5 mins of human videos. 🧵↓

Jingyun Yang

88,352 views • 2 years ago

UNREAL ENGINE 5 DEVS, THIS IS A MUST WATCH! As you guys know, I always try to bring the latest news and info for Unreal Engine 5. I found a plugin that allows you to CREATE BLUEPRINTS AND 3D MODELS INSIDE of Unreal Engine 5, this is not a sponsored video. Of course, I recommend using it as a helper and guide, not a robot that will do everything for you etc... enjoy! #LudusAI #ue5 #unrealengine #unrealengine5 #gamedev

UNREAL ENGINE 5 DEVS, THIS IS A MUST WATCH! As you guys know, I always try to bring the latest news and info for Unreal Engine 5. I found a plugin that allows you to CREATE BLUEPRINTS AND 3D MODELS INSIDE of Unreal Engine 5, this is not a sponsored video. Of course, I recommend using it as a helper and guide, not a robot that will do everything for you etc... enjoy! #LudusAI #ue5 #unrealengine #unrealengine5 #gamedev

Gorka Games

35,996 views • 1 year ago

Collecting a high quality dataset with 4M unique phrases and 52M corresponding object masks helped SAM 3 achieve 2x the performance of baseline models. Kate, a researcher on SAM 3, explains how the data engine made this leap possible. 🔗 Read the SAM 3 research paper:

Collecting a high quality dataset with 4M unique phrases and 52M corresponding object masks helped SAM 3 achieve 2x the performance of baseline models. Kate, a researcher on SAM 3, explains how the data engine made this leap possible. 🔗 Read the SAM 3 research paper:

AI at Meta

38,684 views • 7 months ago

A precision robot was filmed drawing a full-scale engine blueprint entirely on its own, showcasing the accuracy and control of modern mechanical plotting systems.🤖✍️

A precision robot was filmed drawing a full-scale engine blueprint entirely on its own, showcasing the accuracy and control of modern mechanical plotting systems.🤖✍️

Ahib Jane

40,591 views • 7 months ago