Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Introduce EgoInfinity: a web-scale (14.6 yrs, 142M clips) data engine that automatically lifts Youtube videos into 4D hand-object interaction, and retargets for robot learning Not only a dataset, but a modular, upgradable engine HF Space:

Vector Wang

2,529 subscribers

108,164 görüntüleme • 4 ay önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

20 Yorum

Vector Wang profil fotoğrafı

Vector Wang4 ay önce

The dataset is open to anyone to use and contribute. The pipeline is modular and upgradable. Come scale things up even without any hardwares! Project Page: Paper: Codes: Dataset:

Damian @ The Lighthouse 🤖💡🏠 profil fotoğrafı

Damian @ The Lighthouse 🤖💡🏠4 ay önce

@antoinemarcel just shared this in our gc. Looks great. Passing this on to a handful of people now.

Jerry Chéng profil fotoğrafı

Jerry Chéng4 ay önce

Absolute game changer! Thanks for the open-sourcing

Vector Wang profil fotoğrafı

Vector Wang4 ay önce

Thanks Jerry! Let’s build it together!

Michael Cho - Rbt/Acc profil fotoğrafı

Michael Cho - Rbt/Acc4 ay önce

Mind blown! Tks for putting this together!!

Vector Wang profil fotoğrafı

Vector Wang4 ay önce

Just a start! As a modular platform, it still can be and needs to be upgraded and improved by us and everyone in the community! Just like XLeRobot, I am just offering a path for everyone to build upon.

ahad profil fotoğrafı

ahad4 ay önce

fire!!!

clankr profil fotoğrafı

clankr4 ay önce

The demos look fantastic. Creating an SE(3)-compliant coordinate space for every camera angle is the key. Here is our take:

Damian @ The Lighthouse 🤖💡🏠 profil fotoğrafı

Damian @ The Lighthouse 🤖💡🏠4 ay önce

@nicks_robots

Tolu profil fotoğrafı

Tolu4 ay önce

❤️

Damian @ The Lighthouse 🤖💡🏠 profil fotoğrafı

Damian @ The Lighthouse 🤖💡🏠4 ay önce

@astridwilde

Damian @ The Lighthouse 🤖💡🏠 profil fotoğrafı

Damian @ The Lighthouse 🤖💡🏠4 ay önce

@grantg07

wasabee_dai profil fotoğrafı

wasabee_dai4 ay önce

Ming blowing, it's a game changer! Thank you for it

Mahid profil fotoğrafı

Mahid4 ay önce

NIce work! How long did you work on this?

🐱🐱💥 profil fotoğrafı

🐱🐱💥4 ay önce

Nice work! Super excited to see what comes ahead!

bnw profil fotoğrafı

bnw4 ay önce

vaporware or fr? most such repos hide a critical component, are you fr fr?

Damian @ The Lighthouse 🤖💡🏠 profil fotoğrafı

Damian @ The Lighthouse 🤖💡🏠4 ay önce

@krithikas12

Milton Calderon Donefer profil fotoğrafı

Milton Calderon Donefer4 ay önce

very interesting!

Kavin Raj profil fotoğrafı

Kavin Raj4 ay önce

amazing work!

Alen Gabs profil fotoğrafı

Alen Gabs4 ay önce

@tommiekerssies

Benzer Videolar

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 görüntüleme • 1 yıl önce

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

Julen Urain

42,451 görüntüleme • 2 yıl önce

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

mirrash.eth 🦇🔊

302,374 görüntüleme • 4 ay önce

Introducing Robo360 dataset 🚀, the first real-world omnispective multi-view and multi-material robotic manipulation dataset. Robo360 captures synchronized multi-modal robot-object interaction data (video, audio, proprioception, control) to facilitate research in dynamic novel-view synthesis, robot learning, physics understanding, world models, and more! Code and data coming soon!

Introducing Robo360 dataset 🚀, the first real-world omnispective multi-view and multi-material robotic manipulation dataset. Robo360 captures synchronized multi-modal robot-object interaction data (video, audio, proprioception, control) to facilitate research in dynamic novel-view synthesis, robot learning, physics understanding, world models, and more! Code and data coming soon!

Litian Liang

23,477 görüntüleme • 2 yıl önce

Sikorsky H-34 unusual engine seen here in a video by AviationWalkarounds on YouTube. Unusual not only for its forward placement on an Helicopter, but also for being a radial engine.

Sikorsky H-34 unusual engine seen here in a video by AviationWalkarounds on YouTube. Unusual not only for its forward placement on an Helicopter, but also for being a radial engine.

Air Safety #OTD by Francisco Cunha

21,491 görüntüleme • 8 ay önce

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without *any* robot teleop data collection 1/n

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without any robot teleop data collection 1/n

Homanga Bharadhwaj @ CVPR

28,408 görüntüleme • 1 yıl önce

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 görüntüleme • 1 yıl önce

We built a robotics data engine with Astra-level performance at 125x lower cost. And we are open-sourcing the weights today! Introducing Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals. Flora already powers our in-house data engine, labeling 200K hours of videos a day on 500 GPUs. We created it to scale the frontier for robot learning. Now you own the weights too. Flora gives you: - High-quality, timestamped captions to curate and filter human egocentric and robot data; - Dense progress rewards for quality control and RL post-training; - Rubrics scoring for real-world benchmarking Try it today, links in thread:

We built a robotics data engine with Astra-level performance at 125x lower cost. And we are open-sourcing the weights today! Introducing Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals. Flora already powers our in-house data engine, labeling 200K hours of videos a day on 500 GPUs. We created it to scale the frontier for robot learning. Now you own the weights too. Flora gives you: - High-quality, timestamped captions to curate and filter human egocentric and robot data; - Dense progress rewards for quality control and RL post-training; - Rubrics scoring for real-world benchmarking Try it today, links in thread:

Jim Fan

161,827 görüntüleme • 24 gün önce

We can convert human videos to robot hand-object interaction trajectories in 4D. Enjoy! Paper: Website: Code: Authors:Bhawna Paliwal,Haritheja,Will Liang, Pieter Abbeel , Mahi Shafiullah 🏠🤖 , Jitendra MALIK

We can convert human videos to robot hand-object interaction trajectories in 4D. Enjoy! Paper: Website: Code: Authors:Bhawna Paliwal,Haritheja,Will Liang, Pieter Abbeel , Mahi Shafiullah 🏠🤖 , Jitendra MALIK

Jitendra MALIK

73,877 görüntüleme • 4 ay önce

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

HumanX: a scalable framework that converts single monocular human videos into agile, generalizable interaction skills for humanoid robots without task-specific rewards. Core parts: - XGen: retargets human motion + synthesizes diverse physically plausible training data (via physics-driven object trajectories and augmentation). - XMimic: unified imitation pipeline for robust generalization. Shows promising results for both Blind (no external sensors) and MoCap modes.

The Humanoid Hub

20,358 görüntüleme • 9 ay önce

Perhaps the "GPT-3 moment" for humanoid robots lies within vast amounts of human video data. UC Berkeley (Abbeel + Malik group) just released Do as I Do: a pipeline that takes monocular RGB human videos (ego + exo)->reconstructs hand-object interactions->retargets them to multi-fingered robotic hands, generating executable robot trajectories. It pushes success rates on real-world clips from ~25% to 71%, outperforming prior SOTA. This kind of human video->robot-complete data approach feels increasingly important for closing the data flywheel in humanoid/embodied AI.

Perhaps the "GPT-3 moment" for humanoid robots lies within vast amounts of human video data. UC Berkeley (Abbeel + Malik group) just released Do as I Do: a pipeline that takes monocular RGB human videos (ego + exo)->reconstructs hand-object interactions->retargets them to multi-fingered robotic hands, generating executable robot trajectories. It pushes success rates on real-world clips from ~25% to 71%, outperforming prior SOTA. This kind of human video->robot-complete data approach feels increasingly important for closing the data flywheel in humanoid/embodied AI.

CyberRobo

12,107 görüntüleme • 4 ay önce

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

MrNeRF

42,688 görüntüleme • 1 yıl önce

🔥🇳🇬💪🏾He don’t need no doggone engine jack! Bro power lifts that engine like it’s a case of beer! Straight up boss! He IS the engine!

🔥🇳🇬💪🏾He don’t need no doggone engine jack! Bro power lifts that engine like it’s a case of beer! Straight up boss! He IS the engine!

Mario Nawfal

125,825 görüntüleme • 2 ay önce

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,604 görüntüleme • 1 yıl önce

Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior. We’re releasing GRAIL: — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)

Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior. We’re releasing GRAIL: — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)

Ye Yuan

44,468 görüntüleme • 5 ay önce

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Felix Heide

27,853 görüntüleme • 1 yıl önce

A hydraulic lifter is a small but powerful component that plays a huge role in keeping your engine smooth, quiet, and efficient. It sits between the camshaft and the engine's valves, automatically maintaining the perfect valve clearance while the engine runs.

A hydraulic lifter is a small but powerful component that plays a huge role in keeping your engine smooth, quiet, and efficient. It sits between the camshaft and the engine's valves, automatically maintaining the perfect valve clearance while the engine runs.

Mechanical Knowledge

11,373 görüntüleme • 8 ay önce