Loading video...

Video Failed to Load

Go Home

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific...

pascal_bornet's profile picture

Pascal Bornet

123,660 subscribers

185,867 views • 2 months ago •via X (Twitter)

News & PoliticsScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:
0:29

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,804 views • 4 months ago

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER
0:30

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 views • 1 year ago

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:
0:35

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

32,025 views • 4 months ago

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)
0:41

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,336 views • 1 year ago

Robotics is still data starved. Collecting high-quality robot demonstrations remains brutally slow and expensive. Introducing COBALT: A cloud-native teleoperation platform designed for large-scale robot learning. We are democratizing data collection by leveraging the hardware everyone already owns: the smartphone All you need is to download an app (today)! Read on for more!
0:39

Robotics is still data starved. Collecting high-quality robot demonstrations remains brutally slow and expensive. Introducing COBALT: A cloud-native teleoperation platform designed for large-scale robot learning. We are democratizing data collection by leveraging the hardware everyone already owns: the smartphone All you need is to download an app (today)! Read on for more!

Animesh Garg

100,693 views • 22 days ago

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)
1:16

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

Cheng Chi

438,741 views • 2 years ago

.Sureform (YC X25) connects real-world workplaces with robotics labs to collect task-specific training data for robot foundation models. Congrats on the launch, Ananth Kashyap!
0:34

.Sureform (YC X25) connects real-world workplaces with robotics labs to collect task-specific training data for robot foundation models. Congrats on the launch, Ananth Kashyap!

Y Combinator

35,876 views • 3 months ago

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵
0:55

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Mahi Shafiullah 🏠🤖

84,848 views • 3 days ago

Humanoid robotics is entering a new phase： Xiaomi Robotics 0 (XR-0) explores learning manipulation directly from large-scale human videos — aligning video understanding with robot embodiment to generate executable actions. Watch humans->learn representations ->map to robot control. Robot data is scarce. Internet video is not. If video pretraining transfers reliably into embodied control, the scaling law of robotics changes. Deployment speed becomes a function of model alignment, not just hardware iteration. We’re moving from engineered skills to learned behavior. That’s a structural shift. project:
0:52

Humanoid robotics is entering a new phase： Xiaomi Robotics 0 (XR-0) explores learning manipulation directly from large-scale human videos — aligning video understanding with robot embodiment to generate executable actions. Watch humans->learn representations ->map to robot control. Robot data is scarce. Internet video is not. If video pretraining transfers reliably into embodied control, the scaling law of robotics changes. Deployment speed becomes a function of model alignment, not just hardware iteration. We’re moving from engineered skills to learned behavior. That’s a structural shift. project:

CyberRobo

18,356 views • 4 months ago

The Memo robot by Sunday Robotics autonomously stacks and folds socks into a tucked bundle. The robot is operated by the ACT-1 foundation model built in-house at Sunday. It learns from data captured by humans wearing gloves that match Memo's hands.
0:48

The Memo robot by Sunday Robotics autonomously stacks and folds socks into a tucked bundle. The robot is operated by the ACT-1 foundation model built in-house at Sunday. It learns from data captured by humans wearing gloves that match Memo's hands.

The Humanoid Hub

66,547 views • 7 months ago

Axis is officially LIVE on Base. 🔵 Axis is scaling Physical AI for the real world, contributed by everyone. You can control robots in a virtual world, generate training data at scale, and help build the brain behind tomorrow's robots. All from browser. No hardware needed. Start building robotics intelligence today:
0:41

Axis is officially LIVE on Base. 🔵 Axis is scaling Physical AI for the real world, contributed by everyone. You can control robots in a virtual world, generate training data at scale, and help build the brain behind tomorrow's robots. All from browser. No hardware needed. Start building robotics intelligence today:

Axis Robotics

230,701 views • 3 months ago

1/ The data parsing market for robotics is just emerging, and it’s already valued in the tens of billions. Data parsing to build data models and software for Robotics is a $20B market in 2024 and is expected to reach $150B+ by 2034 Cherry AI Robotics has built a data parsing tool for robotics, which goes live next week, and will effectively make this project one of the pioneers in data parsing for robotics.
0:25

1/ The data parsing market for robotics is just emerging, and it’s already valued in the tens of billions. Data parsing to build data models and software for Robotics is a $20B market in 2024 and is expected to reach $150B+ by 2034 Cherry AI Robotics has built a data parsing tool for robotics, which goes live next week, and will effectively make this project one of the pioneers in data parsing for robotics.

CherryAI Robotics

90,786 views • 9 months ago

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.
0:50

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.

The Humanoid Hub

44,441 views • 3 months ago

Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog →
1:39

Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog →

LeRobot

280,565 views • 2 months ago

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.
2:05

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 views • 11 months ago

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.
1:14

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 views • 4 months ago

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.
44:56

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.

RoboPapers

88,996 views • 8 months ago

1/ Cherry AI Robotics is building the world’s first data layer for robotics. Every autonomous robot needs a neural network to operate, which requires massive domain-specific data. Cherry AI Robotics has an entire ecosystem of products with 200k+ communities and 800k+ users. These products generate large amounts of data, which is parsed to create valuable data sets for robotics. Learn more:
0:30

1/ Cherry AI Robotics is building the world’s first data layer for robotics. Every autonomous robot needs a neural network to operate, which requires massive domain-specific data. Cherry AI Robotics has an entire ecosystem of products with 200k+ communities and 800k+ users. These products generate large amounts of data, which is parsed to create valuable data sets for robotics. Learn more:

CherryAI Robotics

105,357 views • 9 months ago

the micro1 robotics lab: real world data for intelligent models that co-exist in the physical world. we’re in-the-wild across 75 countries in 6,000+ unique environments collecting data. diverse movements, objects, and settings. the future of AI is as human as you can imagine. join us to start training robots today (link in comments).
1:05

the micro1 robotics lab: real world data for intelligent models that co-exist in the physical world. we’re in-the-wild across 75 countries in 6,000+ unique environments collecting data. diverse movements, objects, and settings. the future of AI is as human as you can imagine. join us to start training robots today (link in comments).

Ali Ansari

207,078 views • 3 months ago