Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without any robot teleop data collection 1/n

Homanga Bharadhwaj @ CVPR

3,024 subscribers

28,408 Aufrufe • vor 1 Jahr •via X (Twitter)

Gesundheit & Wellness Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

4 Kommentare

Profilbild von Homanga @ CVPR

Homanga @ CVPRvor 1 Jahr

To train a functional grasp prediction policy: -> we create a dataset of robot-object grasps by re-targeting human hand-object interactions from web images followed by object mesh refinement with a text-to-3D model -> perform imitation learning on the resulting dataset! 2/n

Profilbild von Homanga @ CVPR

Homanga @ CVPRvor 1 Jahr

Web2Grasp was led by amazing @CMU_Robotics collaborators @chen_hongyi_ and @YaoYunchao Check out Hongyi's thread below and the website for more details n/n

Profilbild von AirFranz

AirFranzvor 1 Jahr

Hey Homanga! Great work. Just DM’ed you. Franz

Profilbild von Craig ⚔️

Craig ⚔️vor 1 Jahr

Is this 1994 tech?

Ähnliche Videos

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

X Square Robot has introduced TwinDEX: a co-designed pair of three-finger, 9-DoF dexterous hands, one wearable for data collection, one matched robot end effector for deployment. The wearable and the robot hand share the same kinematics, geometry, materials, look, and sensors, so human demos map straight to the robot without retargeting. Human data collection improves throughput 5.3x compared to teleoperation, and a policy can be trained with a few hundred episodes without any robot data.

The Humanoid Hub

277,022 Aufrufe • vor 1 Monat

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

122,225 Aufrufe • vor 1 Jahr

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,358 Aufrufe • vor 1 Jahr

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

Cheng Chi

440,035 Aufrufe • vor 2 Jahren

This dexterous AI-operated robot hand taught itself to manipulate objects

This dexterous AI-operated robot hand taught itself to manipulate objects

Mashable

18,893 Aufrufe • vor 3 Jahren

Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids. - learned directly from human manipulation data - no teleop/robot data - close to human-level dexterity and efficiency - multi-robot collab

Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids. - learned directly from human manipulation data - no teleop/robot data - close to human-level dexterity and efficiency - multi-robot collab

Reward AI

1,774,160 Aufrufe • vor 1 Monat

These days, it feels like a new robotic hand is developed every week. But how can we give them all vision-based general grasping ability? Meet AnyDexGrasp: it enables robust grasping across different robot hands—with just 40 training objects and 4–8 hours of trial and error!

These days, it feels like a new robotic hand is developed every week. But how can we give them all vision-based general grasping ability? Meet AnyDexGrasp: it enables robust grasping across different robot hands—with just 40 training objects and 4–8 hours of trial and error!

Hao-Shu Fang

25,409 Aufrufe • vor 1 Jahr

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N

What representation enables open-world robot manipulation from generated videos? Introducing Dream2Flow, our recent work that bridges video generation and robot control with 3D object flow. Stanford University #ICRA2026 1/N

Wenlong Huang

109,233 Aufrufe • vor 7 Monaten

In-hand object manipulation is a dexterity litmus test for robot hands. Our new system in Science Robotics Science Robotics can dynamically reorient many different objects in hand in the air. 📚Project website: 🧑‍💻Code: 🧵1/n

In-hand object manipulation is a dexterity litmus test for robot hands. Our new system in Science Robotics Science Robotics can dynamically reorient many different objects in hand in the air. 📚Project website: 🧑‍💻Code: 🧵1/n

Tao Chen

30,724 Aufrufe • vor 2 Jahren

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Chen Wang

239,752 Aufrufe • vor 2 Jahren

Low-cost teleop systems have democratized robot data collection, but they lack any force feedback, making it challenging to teleoperate contact-rich tasks. Many robot arms provide force information — a critical yet underutilized modality in robot learning. We introduce: 1. 🦾A low-cost, force-feedback-enabled teleop system. 2. 🥊Force-Attending Curriculum Training (FACTR) uses force to improve generalization in complex, contact-rich tasks. 🧵(1/N)

Low-cost teleop systems have democratized robot data collection, but they lack any force feedback, making it challenging to teleoperate contact-rich tasks. Many robot arms provide force information — a critical yet underutilized modality in robot learning. We introduce: 1. 🦾A low-cost, force-feedback-enabled teleop system. 2. 🥊Force-Attending Curriculum Training (FACTR) uses force to improve generalization in complex, contact-rich tasks. 🧵(1/N)

Jason Liu

152,482 Aufrufe • vor 1 Jahr

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:

Zhengyi “Zen” Luo

92,214 Aufrufe • vor 2 Jahren

Mimic Robotics released a full-stack dexterity platform: the mimic hand M1 + umimic (U1) wearable. They keep human hand morphology consistent across the entire data-model-dexterous hand-manipulation pipeline--from video pre-training to wearable demos and robot deployment. This avoids cross-embodiment gaps and captures the essence of upper-limb intelligence. M1: Tendon-driven (15 active DoF), highly backdrivable ( 25kg grasp, industrial-grade, Swiss-made. U1 lets humans teach the robot naturally with their own hands--1:1 kinematics and sensing match, no latency or retargeting, enabling higher-quality and more scalable data. Looking forward to its performance at the customer site.

Mimic Robotics released a full-stack dexterity platform: the mimic hand M1 + umimic (U1) wearable. They keep human hand morphology consistent across the entire data-model-dexterous hand-manipulation pipeline--from video pre-training to wearable demos and robot deployment. This avoids cross-embodiment gaps and captures the essence of upper-limb intelligence. M1: Tendon-driven (15 active DoF), highly backdrivable ( 25kg grasp, industrial-grade, Swiss-made. U1 lets humans teach the robot naturally with their own hands--1:1 kinematics and sensing match, no latency or retargeting, enabling higher-quality and more scalable data. Looking forward to its performance at the customer site.

CyberRobo

26,160 Aufrufe • vor 3 Monaten

We talked to Ritvik Singh about how you can train sim-to-real dexterous manipulation policies using NVIDIA Isaac. This robot is grasping objects using pure RGB stereo: take in images from a camera pair and predict what to do, all without training in the real world.

We talked to Ritvik Singh about how you can train sim-to-real dexterous manipulation policies using NVIDIA Isaac. This robot is grasping objects using pure RGB stereo: take in images from a camera pair and predict what to do, all without training in the real world.

Chris Paxton

20,067 Aufrufe • vor 1 Jahr

Thrilled to share one of my favorite works this year: DexNDM! We bridge the Sim2Real gap for dexterous in-hand rotation, achieving a true "0-to-1" advancement. The key? DexNDM learns from biased, real-world data without needing any successful demonstrations. Now a general-purpose dexterous hand can stably rotate large books, long rods, & complex objects around any axis, from any wrist pose. This powerful primitive enables complex, long-horizon tasks like teleoperated screwing and furniture assembly. 📄 Paper: 🌐 Project Page:

Thrilled to share one of my favorite works this year: DexNDM! We bridge the Sim2Real gap for dexterous in-hand rotation, achieving a true "0-to-1" advancement. The key? DexNDM learns from biased, real-world data without needing any successful demonstrations. Now a general-purpose dexterous hand can stably rotate large books, long rods, & complex objects around any axis, from any wrist pose. This powerful primitive enables complex, long-horizon tasks like teleoperated screwing and furniture assembly. 📄 Paper: 🌐 Project Page:

Li Yi

38,969 Aufrufe • vor 11 Monaten

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 Aufrufe • vor 2 Jahren

Reward AI just came out of stealth and unveiled OM-1, a robot policy learned entirely from humans. No teleoperation, no on-robot data. One Model, Any Body. 1. Capture. A wearable 7-DoF hand (Omnibody Hand) lets people demonstrate manipulation naturally while working, cooking, or sorting. Sensors record tactile, proximity, force and hand pose at full human speed. 2. Learn. OM-1 trains on this human data alone. No teleop, no on-robot experience. Then it runs across bodies from industrial arms to humanoids, and scaling is just adding more human data. 3. Control. A high-frequency control layer, trained with RL in sim, executes those actions on any robot, holding steady through unknown loads like a stuck fridge door. The result: OM-1 picks up a brand-new task from under 30 minutes of data.

The Humanoid Hub

35,764 Aufrufe • vor 1 Monat

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

Boston Dynamics collaborated with NVIDIA to demonstrate DextrAH-RGB, a workflow for dexterous grasping from stereo RGB input. The end-to-end policy for Atlas robot, trained entirely in NVIDIA Isaac Lab, transfers zero-shot from simulation to the real robot.

The Humanoid Hub

79,781 Aufrufe • vor 1 Jahr

Imagine if you have an object in hand, you can rotate the object by feeling without even looking. This is what we enable the robot to do now: Rotating without Seeing. Our multi-finger robot hand learns to rotate diverse objects using only touch sensing.

Imagine if you have an object in hand, you can rotate the object by feeling without even looking. This is what we enable the robot to do now: Rotating without Seeing. Our multi-finger robot hand learns to rotate diverse objects using only touch sensing.

Xiaolong Wang

65,293 Aufrufe • vor 3 Jahren

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

X Square Robot

1,346,470 Aufrufe • vor 1 Monat