Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Introduce EgoInfinity: a web-scale (14.6 yrs, 142M clips) data engine that automatically lifts Youtube videos into 4D hand-object interaction, and retargets for robot learning Not only a dataset, but a modular, upgradable engine HF Space:

Vector Wang

2,529 subscribers

108,164 просмотров • 4 месяцев назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 20

Фото профиля Vector Wang

Vector Wang4 месяцев назад

The dataset is open to anyone to use and contribute. The pipeline is modular and upgradable. Come scale things up even without any hardwares! Project Page: Paper: Codes: Dataset:

Фото профиля Damian @ The Lighthouse 🤖💡🏠

Damian @ The Lighthouse 🤖💡🏠4 месяцев назад

@antoinemarcel just shared this in our gc. Looks great. Passing this on to a handful of people now.

Фото профиля Jerry Chéng

Jerry Chéng4 месяцев назад

Absolute game changer! Thanks for the open-sourcing

Фото профиля Vector Wang

Vector Wang4 месяцев назад

Thanks Jerry! Let’s build it together!

Фото профиля Michael Cho - Rbt/Acc

Michael Cho - Rbt/Acc4 месяцев назад

Mind blown! Tks for putting this together!!

Фото профиля Vector Wang

Vector Wang4 месяцев назад

Just a start! As a modular platform, it still can be and needs to be upgraded and improved by us and everyone in the community! Just like XLeRobot, I am just offering a path for everyone to build upon.

Фото профиля ahad

ahad4 месяцев назад

fire!!!

Фото профиля clankr

clankr4 месяцев назад

The demos look fantastic. Creating an SE(3)-compliant coordinate space for every camera angle is the key. Here is our take:

Фото профиля Damian @ The Lighthouse 🤖💡🏠

Damian @ The Lighthouse 🤖💡🏠4 месяцев назад

@nicks_robots

Фото профиля Tolu

Tolu4 месяцев назад

❤️

Фото профиля Damian @ The Lighthouse 🤖💡🏠

Damian @ The Lighthouse 🤖💡🏠4 месяцев назад

@astridwilde

Фото профиля Damian @ The Lighthouse 🤖💡🏠

Damian @ The Lighthouse 🤖💡🏠4 месяцев назад

@grantg07

Фото профиля wasabee_dai

wasabee_dai4 месяцев назад

Ming blowing, it's a game changer! Thank you for it

Фото профиля Mahid

Mahid4 месяцев назад

NIce work! How long did you work on this?

Фото профиля 🐱🐱💥

🐱🐱💥4 месяцев назад

Nice work! Super excited to see what comes ahead!

Фото профиля bnw

bnw4 месяцев назад

vaporware or fr? most such repos hide a critical component, are you fr fr?

Фото профиля Damian @ The Lighthouse 🤖💡🏠

Damian @ The Lighthouse 🤖💡🏠4 месяцев назад

@krithikas12

Фото профиля Milton Calderon Donefer

Milton Calderon Donefer4 месяцев назад

very interesting!

Фото профиля Kavin Raj

Kavin Raj4 месяцев назад

amazing work!

Фото профиля Alen Gabs

Alen Gabs4 месяцев назад

@tommiekerssies

Похожие видео

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 просмотров • 1 год назад

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

Julen Urain

42,451 просмотров • 2 лет назад

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

mirrash.eth 🦇🔊

302,374 просмотров • 4 месяцев назад

Introducing Robo360 dataset 🚀, the first real-world omnispective multi-view and multi-material robotic manipulation dataset. Robo360 captures synchronized multi-modal robot-object interaction data (video, audio, proprioception, control) to facilitate research in dynamic novel-view synthesis, robot learning, physics understanding, world models, and more! Code and data coming soon!

Introducing Robo360 dataset 🚀, the first real-world omnispective multi-view and multi-material robotic manipulation dataset. Robo360 captures synchronized multi-modal robot-object interaction data (video, audio, proprioception, control) to facilitate research in dynamic novel-view synthesis, robot learning, physics understanding, world models, and more! Code and data coming soon!

Litian Liang

23,477 просмотров • 2 лет назад

Sikorsky H-34 unusual engine seen here in a video by AviationWalkarounds on YouTube. Unusual not only for its forward placement on an Helicopter, but also for being a radial engine.

Sikorsky H-34 unusual engine seen here in a video by AviationWalkarounds on YouTube. Unusual not only for its forward placement on an Helicopter, but also for being a radial engine.

Air Safety #OTD by Francisco Cunha

21,491 просмотров • 8 месяцев назад

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without *any* robot teleop data collection 1/n

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without any robot teleop data collection 1/n

Homanga Bharadhwaj @ CVPR

28,408 просмотров • 1 год назад

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 просмотров • 1 год назад

We built a robotics data engine with Astra-level performance at 125x lower cost. And we are open-sourcing the weights today! Introducing Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals. Flora already powers our in-house data engine, labeling 200K hours of videos a day on 500 GPUs. We created it to scale the frontier for robot learning. Now you own the weights too. Flora gives you: - High-quality, timestamped captions to curate and filter human egocentric and robot data; - Dense progress rewards for quality control and RL post-training; - Rubrics scoring for real-world benchmarking Try it today, links in thread:

We built a robotics data engine with Astra-level performance at 125x lower cost. And we are open-sourcing the weights today! Introducing Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals. Flora already powers our in-house data engine, labeling 200K hours of videos a day on 500 GPUs. We created it to scale the frontier for robot learning. Now you own the weights too. Flora gives you: - High-quality, timestamped captions to curate and filter human egocentric and robot data; - Dense progress rewards for quality control and RL post-training; - Rubrics scoring for real-world benchmarking Try it today, links in thread:

Jim Fan

161,827 просмотров • 24 дней назад

We can convert human videos to robot hand-object interaction trajectories in 4D. Enjoy! Paper: Website: Code: Authors:Bhawna Paliwal,Haritheja,Will Liang, Pieter Abbeel , Mahi Shafiullah 🏠🤖 , Jitendra MALIK

We can convert human videos to robot hand-object interaction trajectories in 4D. Enjoy! Paper: Website: Code: Authors:Bhawna Paliwal,Haritheja,Will Liang, Pieter Abbeel , Mahi Shafiullah 🏠🤖 , Jitendra MALIK

Jitendra MALIK

73,877 просмотров • 4 месяцев назад

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

The Humanoid Hub

20,358 просмотров • 9 месяцев назад

Perhaps the "GPT-3 moment" for humanoid robots lies within vast amounts of human video data. UC Berkeley (Abbeel + Malik group) just released Do as I Do: a pipeline that takes monocular RGB human videos (ego + exo)->reconstructs hand-object interactions->retargets them to multi-fingered robotic hands, generating executable robot trajectories. It pushes success rates on real-world clips from ~25% to 71%, outperforming prior SOTA. This kind of human video->robot-complete data approach feels increasingly important for closing the data flywheel in humanoid/embodied AI.

Perhaps the "GPT-3 moment" for humanoid robots lies within vast amounts of human video data. UC Berkeley (Abbeel + Malik group) just released Do as I Do: a pipeline that takes monocular RGB human videos (ego + exo)->reconstructs hand-object interactions->retargets them to multi-fingered robotic hands, generating executable robot trajectories. It pushes success rates on real-world clips from ~25% to 71%, outperforming prior SOTA. This kind of human video->robot-complete data approach feels increasingly important for closing the data flywheel in humanoid/embodied AI.

CyberRobo

12,107 просмотров • 4 месяцев назад

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

MrNeRF

42,688 просмотров • 1 год назад

🔥🇳🇬💪🏾He don’t need no doggone engine jack! Bro power lifts that engine like it’s a case of beer! Straight up boss! He IS the engine!

🔥🇳🇬💪🏾He don’t need no doggone engine jack! Bro power lifts that engine like it’s a case of beer! Straight up boss! He IS the engine!

Mario Nawfal

125,825 просмотров • 2 месяцев назад

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,604 просмотров • 1 год назад

Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior. We’re releasing GRAIL: — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)

Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior. We’re releasing GRAIL: — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)

Ye Yuan

44,468 просмотров • 5 месяцев назад

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Felix Heide

27,853 просмотров • 1 год назад

A hydraulic lifter is a small but powerful component that plays a huge role in keeping your engine smooth, quiet, and efficient. It sits between the camshaft and the engine's valves, automatically maintaining the perfect valve clearance while the engine runs.

A hydraulic lifter is a small but powerful component that plays a huge role in keeping your engine smooth, quiet, and efficient. It sits between the camshaft and the engine's valves, automatically maintaining the perfect valve clearance while the engine runs.

Mechanical Knowledge

11,373 просмотров • 8 месяцев назад