Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

we open sourced the code to transform human videos into robot trajectories, so you can train robots with your hands 👐🏻 we used it in our recent paper R+X: Retrieval and Execution from Everyday Human Videos (ICRA 2025 🇺🇸) link and details in thread 🧵

normandipalo's profile picture

Norman Di Palo

1,518 subscribers

10,578 Aufrufe • vor 1 Jahr •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.
0:14

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.

Ruihan Yang

58,685 Aufrufe • vor 11 Monaten

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.
0:18

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.

Lerrel Pinto

31,143 Aufrufe • vor 23 Tagen

This also shows up in the representations learned by the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.
0:10

This also shows up in the representations learned by the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.

Physical Intelligence

120,595 Aufrufe • vor 6 Monaten

Checkout ObjectForesight! We tackle the problem of predicting future 3D object trajectories given a past context, purely from human videos of everyday interactions! [1/n]
0:15

Checkout ObjectForesight! We tackle the problem of predicting future 3D object trajectories given a past context, purely from human videos of everyday interactions! [1/n]

Homanga Bharadhwaj

20,796 Aufrufe • vor 6 Monaten

“The hand is where the mind meets the world.” — Carl Zimmer Humanoid hands closely replicating human degrees of freedom enable robots to seamlessly integrate into our world and learn more efficiently from human demonstrations. A look at some of the impressive hands from 2024🧵
0:12

“The hand is where the mind meets the world.” — Carl Zimmer Humanoid hands closely replicating human degrees of freedom enable robots to seamlessly integrate into our world and learn more efficiently from human demonstrations. A look at some of the impressive hands from 2024🧵

The Humanoid Hub

43,091 Aufrufe • vor 1 Jahr

Grasps are one of the primary ways in which we interact with and shape our environments. How can we faithfully capture human grasps with details such as hand/object shape and contact points? At #CVPR2026, we present MANUS, a method to accurately reconstruct grasps and contacts. 🧵
0:13

Grasps are one of the primary ways in which we interact with and shape our environments. How can we faithfully capture human grasps with details such as hand/object shape and contact points? At #CVPR2026, we present MANUS, a method to accurately reconstruct grasps and contacts. 🧵

Srinath Sridhar

10,767 Aufrufe • vor 2 Jahren

Robot hands are already becoming more powerful than human hands. The next big AGI moment is when we can buy fancy robots for $100K to do all the housework!
0:15

Robot hands are already becoming more powerful than human hands. The next big AGI moment is when we can buy fancy robots for $100K to do all the housework!

Bindu Reddy

152,803 Aufrufe • vor 10 Monaten

X just open-sourced their algorithm code, and now you know how to position your content for max visibility. 10 keys to make your posts blow up, in this thread 🧵 1. Video is king, gets a 1.8x boost 👇
0:10

X just open-sourced their algorithm code, and now you know how to position your content for max visibility. 10 keys to make your posts blow up, in this thread 🧵 1. Video is king, gets a 1.8x boost 👇

TechHalla

15,802 Aufrufe • vor 10 Monaten

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.
0:10

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.

MrNeRF

24,729 Aufrufe • vor 11 Monaten

We have released the code and weights for our #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:
0:14

We have released the code and weights for our #CVPR2023 paper "Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model"! code: abs: project: The demo is below:

Artsiom Sanakoyeu

35,711 Aufrufe • vor 3 Jahren

Don't you have a haptic chair for VR gaming? Don't worry, your friendly humanoid robot can help you with that! In a recent research paper from the University of Colorado, researchers experimented with using a humanoid robot to literally grab your chair and move it during VR racing sessions to make the game more realistic. And from the users' feedback, it seemed to work. I guess they took inspiration from those viral videos, where in some low-budget arcades, the owner literally moved the station with his own hands. I can't wait for the fail videos caused by this technology... Full video is available here: #VirtualReality #VR #technology
0:17

Don't you have a haptic chair for VR gaming? Don't worry, your friendly humanoid robot can help you with that! In a recent research paper from the University of Colorado, researchers experimented with using a humanoid robot to literally grab your chair and move it during VR racing sessions to make the game more realistic. And from the users' feedback, it seemed to work. I guess they took inspiration from those viral videos, where in some low-budget arcades, the owner literally moved the station with his own hands. I can't wait for the fail videos caused by this technology... Full video is available here: #VirtualReality #VR #technology

TonyVT SkarredGhost

80,363 Aufrufe • vor 2 Monaten

Excited to share our SIGGRAPH Asia 2024 paper GVHMR 🥳. We can accurately estimates human motions in the world coordinate system, which are readily transferred to the simulator. Code: HuggingFace demo:
0:16

Excited to share our SIGGRAPH Asia 2024 paper GVHMR 🥳. We can accurately estimates human motions in the world coordinate system, which are readily transferred to the simulator. Code: HuggingFace demo:

Sida Peng

33,548 Aufrufe • vor 1 Jahr

A FAQ thread about Horiza Horiza | Beyond DeFi We always get questions from the community, and today we address them in a thread Note that you can ask your questions in the comments! 🧵❓
0:15

A FAQ thread about Horiza Horiza | Beyond DeFi We always get questions from the community, and today we address them in a thread Note that you can ask your questions in the comments! 🧵❓

Horiza | Beyond DeFi

14,094 Aufrufe • vor 2 Jahren

Proud to announce Dobb·E: the next step in home robot system that I was working on for the past 3 years. We have visited 10 homes, learned 100+ tasks, and we are just getting started! And we fully open-sourced it all, hardware, models, and software: 🧵
0:17

Proud to announce Dobb·E: the next step in home robot system that I was working on for the past 3 years. We have visited 10 homes, learned 100+ tasks, and we are just getting started! And we fully open-sourced it all, hardware, models, and software: 🧵

Mahi Shafiullah 🏠🤖

164,452 Aufrufe • vor 2 Jahren

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:
0:18

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

Chenfeng_X

28,288 Aufrufe • vor 1 Jahr

𝗥𝗼𝗯𝗼𝘁𝘀 𝗱𝗼𝗻’𝘁 𝗻𝗲𝗲𝗱 𝗺𝗼𝗿𝗲 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻𝘀. 𝗧𝗵𝗲𝘆 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗳𝗿𝗼𝗺 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 — 𝗮𝗳𝘁𝗲𝗿 𝘄𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝘀. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: 👉 pretrain on human videos 👉 deploy robot policy 👉 observe failures 👉 reinterpret failures using human priors 👉 improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: 📈 40% → 81% success rate 🏆 Strong improvements over π0.6 RECAP and RISE ✔️ Zero human intervention during post-deployment improvement 🧬 Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same π0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. 📄 Paper: 🌐 Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zürich TU München Microsoft Check out Hanzhi's 🧵 for more details
0:12

𝗥𝗼𝗯𝗼𝘁𝘀 𝗱𝗼𝗻’𝘁 𝗻𝗲𝗲𝗱 𝗺𝗼𝗿𝗲 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻𝘀. 𝗧𝗵𝗲𝘆 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗳𝗿𝗼𝗺 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 — 𝗮𝗳𝘁𝗲𝗿 𝘄𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝘀. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: 👉 pretrain on human videos 👉 deploy robot policy 👉 observe failures 👉 reinterpret failures using human priors 👉 improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: 📈 40% → 81% success rate 🏆 Strong improvements over π0.6 RECAP and RISE ✔️ Zero human intervention during post-deployment improvement 🧬 Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same π0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. 📄 Paper: 🌐 Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zürich TU München Microsoft Check out Hanzhi's 🧵 for more details

Oier Mees

11,985 Aufrufe • vor 14 Tagen

HIP MOBILITY - We can all benefit from some added attention to the hips. If you are unsure of where to start… Start with this thread 🧵 Hip swivels, use your hands to keep your posture upright and alternate legs going into internal rotation.
0:15

HIP MOBILITY - We can all benefit from some added attention to the hips. If you are unsure of where to start… Start with this thread 🧵 Hip swivels, use your hands to keep your posture upright and alternate legs going into internal rotation.

Vernon Griffith, CSCS*D

101,285 Aufrufe • vor 1 Jahr

Tesla engineers on the next-gen Optimus V3 hand: "As we go into gen-3 and into mass production, we are getting very close to human functionality and form factor. It won't even look like a robot. It will look like a human in a superhero suit. It will be something revolutionary."
0:19

Tesla engineers on the next-gen Optimus V3 hand: "As we go into gen-3 and into mass production, we are getting very close to human functionality and form factor. It won't even look like a robot. It will look like a human in a superhero suit. It will be something revolutionary."

Sawyer Merritt

511,656 Aufrufe • vor 3 Monaten

Eight artists. Eight Stories. Procreate presents Human Made. We explore human creativity through intimate interviews with artists from across the globe. Human Made kicks off on 19 November on our YouTube channel. With two artist interviews dropping weekly, make sure you subscribe to catch each new episode and other fresh #Procreate content ✨ Use #ProcreateArtist to your own human made artworks so we can discover them!
0:17

Eight artists. Eight Stories. Procreate presents Human Made. We explore human creativity through intimate interviews with artists from across the globe. Human Made kicks off on 19 November on our YouTube channel. With two artist interviews dropping weekly, make sure you subscribe to catch each new episode and other fresh #Procreate content ✨ Use #ProcreateArtist to your own human made artworks so we can discover them!

Procreate

83,928 Aufrufe • vor 1 Jahr

today we launched 🪴Boosts v2🌴 in Arc. a collection of simple tools that let you edit the internet and make it your own (mostly without code!) a thread of my favorite artifacts from this project: early prototypes, fun design details, and deleted scenes 🧵:
0:18

today we launched 🪴Boosts v2🌴 in Arc. a collection of simple tools that let you edit the internet and make it your own (mostly without code!) a thread of my favorite artifacts from this project: early prototypes, fun design details, and deleted scenes 🧵:

nate parrott

153,248 Aufrufe • vor 3 Jahren