Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

We taught a robot to play! Introducing Dexterity from Supervised Play (DSP): turning minutes of human play into a world model and dexterous manipulation policy. No task demonstrations or sim2real!

ruoshi_liu's profile picture

Ruoshi Liu

2,919 subscribers

26,153 görüntüleme • 12 gün önce •via X (Twitter)

Bilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

12 Yorum

Ruoshi Liu profil fotoğrafı
Ruoshi Liu12 gün önce

The core hypothesis of the paper is that play is a behavior that can be modeled as a goal-agnostic policy and thus be learned through demonstrations. Therefore, we first collect a few minutes of play data by teleoperation.

Ruoshi Liu profil fotoğrafı
Ruoshi Liu12 gün önce

And this worked quite well! With only minutes of demonstrations, we learn a play policy that can safely and efficiently collect play data autonomously for hours or more.

Ruoshi Liu profil fotoğrafı
Ruoshi Liu12 gün önce

With hours of play data, we can train a world model to predict the state of the world conditioned on robot actions.

Ruoshi Liu profil fotoğrafı
Ruoshi Liu12 gün önce

With this learned world model, we can perform MPC planning to achieve a specific task. In this case, we managed to achieve any-to-any pose in-hand object reorientation with a 90% success rate (35 deg threshold).

Ruoshi Liu profil fotoğrafı
Ruoshi Liu12 gün önce

Our method is 500x more sample-efficient than real-world RL and have no sim-to-real gap because it's learn directly in the real world.

Ruoshi Liu profil fotoğrafı
Ruoshi Liu12 gün önce

Our method doesn't only apply to multi-fingered hand but all kinds of contact-rich manipulation tasks.

Ruoshi Liu profil fotoğrafı
Ruoshi Liu12 gün önce

This project is led by Botao He (on the job market!), with Yiming Dou, Yiannis Aloimonos, Jitendra Malik, Akash Sharma at Amazon FAR and UMD. Please checkout our website for paper, code, and more details!

/ profil fotoğrafı
/12 gün önce

@JitendraMalikCV @BotaoUMD @akashshrm02 @_YimingDou you guys need some aprilcubes

Nikhil Kumar profil fotoğrafı
Nikhil Kumar12 gün önce

@JitendraMalikCV @BotaoUMD @akashshrm02 @_YimingDou great demo

Nghẹo | DOPE profil fotoğrafı
Nghẹo | DOPE12 gün önce

@JitendraMalikCV @BotaoUMD @akashshrm02 @_YimingDou Turning playful data into real dexterity is impressive

Rui Zhou profil fotoğrafı
Rui Zhou9 gün önce

@JitendraMalikCV @BotaoUMD @akashshrm02 @_YimingDou Learning contact from play first and bringing in the goal at planning time makes a lot of sense, especially for dexterous hands where perfect demos are so hard to collect

aditya bhatia profil fotoğrafı
aditya bhatia11 gün önce

@JitendraMalikCV @BotaoUMD @akashshrm02 @_YimingDou Learning from play without demos or sim2real is a bold direction. In dexterous work the contact events, slip, jamming and regrasp, carry most of the information, so a world model that captures them is valuable.

Benzer Videolar

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D
0:20

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D

Toru

49,892 görüntüleme • 1 yıl önce

Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids. - learned directly from human manipulation data - no teleop/robot data - close to human-level dexterity and efficiency - multi-robot collab
2:19

Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids. - learned directly from human manipulation data - no teleop/robot data - close to human-level dexterity and efficiency - multi-robot collab

Reward AI

1,778,204 görüntüleme • 1 ay önce

Happy to announce mimic-one: a Scalable Model Recipe for General Purpose Robot Dexterity, the culmination of years of research work in dexterous manipulation with imitation learning.
0:29

Happy to announce mimic-one: a Scalable Model Recipe for General Purpose Robot Dexterity, the culmination of years of research work in dexterous manipulation with imitation learning.

Elvis Nava

20,470 görüntüleme • 1 yıl önce

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation
0:27

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

Hongyu Li

106,422 görüntüleme • 1 yıl önce

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!
0:20

Introducing ManipGen: a sim2real agent for zero-shot manipulation. ManipGen handles complex tasks in the real world like organizing shelves, tidying cluttered tables, and more – all from text input and with on human demonstrations!

Deepak Pathak

58,160 görüntüleme • 2 yıl önce

WUJI is showcasing its dexterous robotic hands at IROS 2026, demonstrating how human hand movements can control a robot through glove-based teleoperation. Its WUJI Hand 2 can also perform precise tasks like pen spinning using a policy trained in simulation and transferred to a real robot. These demonstrations highlight advances in robotic finger coordination, dexterity, and fine manipulation.
0:29

WUJI is showcasing its dexterous robotic hands at IROS 2026, demonstrating how human hand movements can control a robot through glove-based teleoperation. Its WUJI Hand 2 can also perform precise tasks like pen spinning using a policy trained in simulation and transferred to a real robot. These demonstrations highlight advances in robotic finger coordination, dexterity, and fine manipulation.

Space and Technology

12,420 görüntüleme • 17 gün önce

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.
0:43

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

122,237 görüntüleme • 1 yıl önce

(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before. The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices. Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL. No demonstrations, no per-task reward tuning. Just PPO at scale. Check out the policies yourself at All videos at 1x speed! 🧵 🔈
1:15

(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before. The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices. Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL. No demonstrations, no per-task reward tuning. Just PPO at scale. Check out the policies yourself at All videos at 1x speed! 🧵 🔈

Octi Zhang

207,140 görüntüleme • 19 gün önce

We trained a robot to play football. How? Self-play for 140 years in a virtual World Cup. Meet the #Messinator:
0:52

We trained a robot to play football. How? Self-play for 140 years in a virtual World Cup. Meet the #Messinator:

Skild AI

1,775,443 görüntüleme • 1 ay önce

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:
2:58

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Lucy Shi

122,827 görüntüleme • 2 yıl önce

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.
0:50

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.

The Humanoid Hub

44,991 görüntüleme • 8 ay önce

Presenting research from Berkeley AI Research Humanoid Intelligence Center. DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation.🖐️🤖 Dexterous manipulation requires touch, yet multi-finger tactile data remain scarce and expensive to collect at scale. Through continual vision-to-touch learning, DexTacWAM adapts a pretrained video world model into a visuo-tactile world model for predictive multi-finger contact modeling and action generation. Across six contact-rich dexterous manipulation tasks, DexTacWAM achieves the highest score on every task, averaging 70.6 vs. 38.0 for the strongest baseline. 🔬 Why not simply inject tactile features into the action policy? With the same multi-finger tactile encoder, policy architecture, and training procedure, replacing the tactile world-model latent with direct tactile features drops the 4-task mean from 74.7 to 26.6. Takeaway: the benefit does not come simply from providing tactile observations to the policy, but from making multi-finger contact evolution part of the predicted world state. All code, model weights, and datasets are now open-sourced! 🌐 Project: 📄 Paper: 💻 Code: 🤗 Weights&Data: Author List: Haoran Yuan, Zekai Wang, Boning Shao Haoran Lu, trevordarrell, Ismini Lourentzou, Wei ZHAN
2:24

Presenting research from Berkeley AI Research Humanoid Intelligence Center. DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation.🖐️🤖 Dexterous manipulation requires touch, yet multi-finger tactile data remain scarce and expensive to collect at scale. Through continual vision-to-touch learning, DexTacWAM adapts a pretrained video world model into a visuo-tactile world model for predictive multi-finger contact modeling and action generation. Across six contact-rich dexterous manipulation tasks, DexTacWAM achieves the highest score on every task, averaging 70.6 vs. 38.0 for the strongest baseline. 🔬 Why not simply inject tactile features into the action policy? With the same multi-finger tactile encoder, policy architecture, and training procedure, replacing the tactile world-model latent with direct tactile features drops the 4-task mean from 74.7 to 26.6. Takeaway: the benefit does not come simply from providing tactile observations to the policy, but from making multi-finger contact evolution part of the predicted world state. All code, model weights, and datasets are now open-sourced! 🌐 Project: 📄 Paper: 💻 Code: 🤗 Weights&Data: Author List: Haoran Yuan, Zekai Wang, Boning Shao Haoran Lu, trevordarrell, Ismini Lourentzou, Wei ZHAN

Berkeley AI Research

113,351 görüntüleme • 29 gün önce

Humanoid robot transforms into a Thanksgiving chef and household butler 👩‍🍳 Of course, these close-ups showcase Sahrpa's dexterous hands and highly flexible, precise manipulation abilities, just like a real human.
1:18

Humanoid robot transforms into a Thanksgiving chef and household butler 👩‍🍳 Of course, these close-ups showcase Sahrpa's dexterous hands and highly flexible, precise manipulation abilities, just like a real human.

CyberRobo

19,632 görüntüleme • 11 ay önce

Jeff Bezos tries the Tactile Telerobot. The Tactile Telerobot is a haptic telerobotic system capable of transmitting realistic touch feedback to an operator located anywhere in the world. Developed through a joint collaboration between Shadow Robot Company, HaptX, and SynTouch, the project’s initial research and development was funded by All Nippon Airways. -- Robotics teams are challenged to turn human-like dexterity from theory into real-world systems. While dexterous manipulation promises major impact across industries, a gap remains between capability and deployment. Shadow Robot helps close this gap through a dexterity-first, structured engagement model that transforms ideas into prototypes and scalable solutions. The "Trusted Hands: Engineering Collaboration with Shadow Robot" whitepaper highlights how collaboration and repeatable frameworks enable organizations to overcome barriers and advance dexterous robotics in new domains. Get it here: #robotics #robots #engineering #technology
0:43

Jeff Bezos tries the Tactile Telerobot. The Tactile Telerobot is a haptic telerobotic system capable of transmitting realistic touch feedback to an operator located anywhere in the world. Developed through a joint collaboration between Shadow Robot Company, HaptX, and SynTouch, the project’s initial research and development was funded by All Nippon Airways. -- Robotics teams are challenged to turn human-like dexterity from theory into real-world systems. While dexterous manipulation promises major impact across industries, a gap remains between capability and deployment. Shadow Robot helps close this gap through a dexterity-first, structured engagement model that transforms ideas into prototypes and scalable solutions. The "Trusted Hands: Engineering Collaboration with Shadow Robot" whitepaper highlights how collaboration and repeatable frameworks enable organizations to overcome barriers and advance dexterous robotics in new domains. Get it here: #robotics #robots #engineering #technology

Wevolver

31,354 görüntüleme • 9 ay önce

Researchers just taught a robot to play tennis. From just clips of a few amateur players performing basic forehands, backhands, and shuffles... ...a robot learned one of the fastest, most coordinated physical skills there is. Insane!
1:09

Researchers just taught a robot to play tennis. From just clips of a few amateur players performing basic forehands, backhands, and shuffles... ...a robot learned one of the fastest, most coordinated physical skills there is. Insane!

Rowan Cheung

39,494 görüntüleme • 6 ay önce

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.
2:48

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.

Skild AI

386,992 görüntüleme • 1 yıl önce

Introducing: Defense of the Agents (DOTA) • 3 lane MOBA • Play as a human on browser or agent on OpenClaw • Fast paced, casual gameplay AI is ushering a new era of indie games where creators build 10x faster and agents also play. Play Now:
0:21

Introducing: Defense of the Agents (DOTA) • 3 lane MOBA • Play as a human on browser or agent on OpenClaw • Fast paced, casual gameplay AI is ushering a new era of indie games where creators build 10x faster and agents also play. Play Now:

AzFlin 🗿

226,149 görüntüleme • 7 ay önce

Another day, another humanoid robot from china AGIBOT introduces GO-1, a generalist foundation model that integrates a vision-language model with a latent planner for enhanced long-horizon and dexterous manipulation.
3:46

Another day, another humanoid robot from china AGIBOT introduces GO-1, a generalist foundation model that integrates a vision-language model with a latent planner for enhanced long-horizon and dexterous manipulation.

Chubby♨️

23,513 görüntüleme • 1 yıl önce