Loading video...

Video Failed to Load

Go Home

Creating a "unified" policy for a robot is uniquely challenging for two reasons: (a) data is scarce; (b) a single robot can only act sequentially. In PolyTask, we learn a single policy to solve many tasks in a lifelong fashion given *just 1 demonstration* for each task. 🧵

LerrelPinto's profile picture

Lerrel Pinto

9,595 subscribers

46,427 views • 2 years ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Evaluating policies on a real robot can be painful. Can we use a world model to get a rough estimate of how good a policy is? Checkout "Evaluating Robot Policies in a World Model". Paper: Demo: Code:
0:10

Evaluating policies on a real robot can be painful. Can we use a world model to get a rough estimate of how good a policy is? Checkout "Evaluating Robot Policies in a World Model". Paper: Demo: Code:

Sherry Yang

36,692 views • 1 year ago

A Rerun Viewer for the DROID Dataset! DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset is a robot manipulation dataset by Alexander Khazatsky et al. with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks.
0:15

A Rerun Viewer for the DROID Dataset! DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset is a robot manipulation dataset by Alexander Khazatsky et al. with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks.

Rerun

21,776 views • 2 years ago

How can a visuomotor policy learn from internet videos? We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:
0:12

How can a visuomotor policy learn from internet videos? We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:

Ruoshi Liu

50,797 views • 2 years ago

Teaching robots how to paint! 💅🏼 This painting robot, for example, mimics human movements with precision to handle repetitive, and tiring tasks. A special device memorizes points in 3D space, which are then sent to the robot's control. The result? A robot that, after a single demonstration, can perform a given action. Keep in mind that it's taught as 'fixed'. 👨🏻‍🔧 That is, it will not be able to react to an anomaly or changing environment. But ultimately, with fewer workers available and many avoiding tough, dirty jobs, robots are helping industries stay efficient and safe. ♻️ RT to help 1 robot find a new workplace!
0:15

Teaching robots how to paint! 💅🏼 This painting robot, for example, mimics human movements with precision to handle repetitive, and tiring tasks. A special device memorizes points in 3D space, which are then sent to the robot's control. The result? A robot that, after a single demonstration, can perform a given action. Keep in mind that it's taught as 'fixed'. 👨🏻‍🔧 That is, it will not be able to react to an anomaly or changing environment. But ultimately, with fewer workers available and many avoiding tough, dirty jobs, robots are helping industries stay efficient and safe. ♻️ RT to help 1 robot find a new workplace!

Lukas Ziegler

38,319 views • 1 year ago

📢#𝗥𝗼𝗯𝗼𝗛𝗶𝘃𝗲 - a unified robot learning framework ✅Designed for genralizn first robot-learning era ✅Diverse (500 envs, 8 domain) ✅Single flag for Sim<>Real ✅TeleOper Support ✅Multi-(Skill x Task) realworld dataset ✅pip install robohive 🧵👇
0:10

📢#𝗥𝗼𝗯𝗼𝗛𝗶𝘃𝗲 - a unified robot learning framework ✅Designed for genralizn first robot-learning era ✅Diverse (500 envs, 8 domain) ✅Single flag for Sim<>Real ✅TeleOper Support ✅Multi-(Skill x Task) realworld dataset ✅pip install robohive 🧵👇

Vikash Kumar

50,718 views • 2 years ago

🤖Why is robot manipulation still an open challenge? This video shows a kitting task -- packing multiple items into a single product. No robot today can do this autonomously. Big challenges = big opportunities for research and industry. #robotics #manipulation #automation
0:19

🤖Why is robot manipulation still an open challenge? This video shows a kitting task -- packing multiple items into a single product. No robot today can do this autonomously. Big challenges = big opportunities for research and industry. #robotics #manipulation #automation

Yu Xiang

21,236 views • 1 year ago

🔥VIRAL: A 24-hour coffee shop run by a single robot opened in Beijing, serving each cup in under 30 seconds. On opening day, it made 202 coffees in one hour, setting a Guinness World Record for a robot-operated store.
0:19

🔥VIRAL: A 24-hour coffee shop run by a single robot opened in Beijing, serving each cup in under 30 seconds. On opening day, it made 202 coffees in one hour, setting a Guinness World Record for a robot-operated store.

Coin Bureau

45,961 views • 8 days ago

Predicting how the environment changes over time is a key task in robotics, enabling all kinds of downstream tasks. Trace Anything is a very interesting look at how you can train models which predict the trajectory each point will take in the future -- you can use this for robot manipulation, video prediction, and more. Learn more ->
0:16

Predicting how the environment changes over time is a key task in robotics, enabling all kinds of downstream tasks. Trace Anything is a very interesting look at how you can train models which predict the trajectory each point will take in the future -- you can use this for robot manipulation, video prediction, and more. Learn more ->

Chris Paxton

56,845 views • 8 months ago

Why is action chunking crucial for robot dexterity? 🤖 - We identify a natural tradeoff between temporal consistency and reactivity - New policy decoding technique that is *both* temporally consistent & fully reactive ICLR 2025 paper: A short thread 🧵
0:10

Why is action chunking crucial for robot dexterity? 🤖 - We identify a natural tradeoff between temporal consistency and reactivity - New policy decoding technique that is *both* temporally consistent & fully reactive ICLR 2025 paper: A short thread 🧵

Chelsea Finn

36,073 views • 1 year ago

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:
0:18

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

Chenfeng_X

28,308 views • 1 year ago

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.
0:18

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.

Lerrel Pinto

33,816 views • 2 months ago

RLDX-1 a dexterity-first foundation model for robot hands, is coming in a few days. It is interesting.
0:18

RLDX-1 a dexterity-first foundation model for robot hands, is coming in a few days. It is interesting.

Brian Roemmele

16,541 views • 3 months ago

Generalist robot policies need a benchmark that works across any robot and any policy. 🦾 Introducing RoboLab, a high‑fidelity simulation benchmark built on NVIDIA Isaac and Omniverse to evaluate generalist robot policies in diverse, photoreal, physics‑based environments. Coming soon to the NVIDIA Isaac Lab‑Arena roadmap for large‑scale, robotic policy evaluation. 📖 #NationalRoboticsWeek
0:15

Generalist robot policies need a benchmark that works across any robot and any policy. 🦾 Introducing RoboLab, a high‑fidelity simulation benchmark built on NVIDIA Isaac and Omniverse to evaluate generalist robot policies in diverse, photoreal, physics‑based environments. Coming soon to the NVIDIA Isaac Lab‑Arena roadmap for large‑scale, robotic policy evaluation. 📖 #NationalRoboticsWeek

NVIDIA Robotics

23,872 views • 4 months ago

US and Israel sending a joint message to Iran: To grasp the sheer power of a B-52 nuclear bomber— Each one can carry 20 B83 bombs, unleashing 2,000 km² of devastation in a single strike. That’s just one bomber. For context, Tehran is only 730 km²
0:18

US and Israel sending a joint message to Iran: To grasp the sheer power of a B-52 nuclear bomber— Each one can carry 20 B83 bombs, unleashing 2,000 km² of devastation in a single strike. That’s just one bomber. For context, Tehran is only 730 km²

Open Source Intel

1,185,252 views • 1 year ago

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.
0:20

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Jianlan Luo

95,100 views • 7 months ago

Check out this robot hexapod with a 25Kg payload that can run for an hour on a single power tool battery. It is fully open source and runs #ROS Jazzy.
0:12

Check out this robot hexapod with a 25Kg payload that can run for an hour on a single power tool battery. It is fully open source and runs #ROS Jazzy.

Open Robotics

12,060 views • 1 month ago

Controlled via a Bluetooth finger patch, BeetleBot, a beetle-inspired soft robot, can perform various locomotive tasks and provide visual and sensory feedback for potential applications from biomedical assistance to environment exploration. Learn more in Science Advances:
0:11

Controlled via a Bluetooth finger patch, BeetleBot, a beetle-inspired soft robot, can perform various locomotive tasks and provide visual and sensory feedback for potential applications from biomedical assistance to environment exploration. Learn more in Science Advances:

Science Magazine

109,914 views • 3 days ago

Trained in Simulation Our robot learns to walk naturally similar to a human via a high fidelity physics simulator We simulate years of data in only a few hours
0:11

Trained in Simulation Our robot learns to walk naturally similar to a human via a high fidelity physics simulator We simulate years of data in only a few hours

Figure

40,689 views • 1 year ago