Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

Chelsea Finn

101,138 subscribers

264,494 görüntüleme • 3 yıl önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Yorum

Chelsea Finn profil fotoğrafı

Chelsea Finn3 yıl önce

First, the hardware: We use simple puppeteering. Just copy the joint angles from the leader to the follower robot. No tactile or force feedback. The manufacturer (@trossenrobotics) didn’t know these tasks were possible.

Chelsea Finn profil fotoğrafı

Chelsea Finn3 yıl önce

Can the robot do all this on its own? We train the robot to predict actions in chunks, rather than one at a time. Recipe: action chunking + transformers + only 50 demonstrations The robot *autonomously* completes fine manipulation skills. ⬇️

Chelsea Finn profil fotoğrafı

Chelsea Finn3 yıl önce

But wait, there’s more! Website & Code: Paper: You can buy the full kit directly from Trossen: Led by the amazing @tonyzzhao, with @svlevine, @Vikashplus

Deepak Pathak profil fotoğrafı

Deepak Pathak3 yıl önce

Great to see it being finally released. Looks very cool. Congrats @chelseabfinn and @tonyzzhao !!

Danijar Hafner profil fotoğrafı

Danijar Hafner3 yıl önce

This looks super cool, congrats!! To what extent do the policies generalize (e.g. object variations, positions)?

Chelsea Finn profil fotoğrafı

Chelsea Finn3 yıl önce

Thanks @danijarh! The demos include ~15cm of object position variation, and the policies generalize to that degree. Need more & wider demo data for more generalization. We also see some generalization to distractors & disturbances.

Co Builder profil fotoğrafı

Co Builder3 yıl önce

This is mind blowing!! 🤩! Congrats!!

Tran Van Phu profil fotoğrafı

Tran Van Phu3 yıl önce

@LukeW I'm having the fastest hosting and utility design. Who needs can contact:

Snm_Str profil fotoğrafı

Snm_Str3 yıl önce

@futuristufuk

Multimodal profil fotoğrafı

Multimodal3 yıl önce

Impressive!

Benzer Videolar

How do we make dexterous hands handle both power and precision tasks with ease? 🫳👌🫰 We introduce Power to Precision (💪➡️🎯), our new paper that optimizes both control and fingertip geometry to unlock robust manipulation from power grasp to fine-grained manipulations. With simplified finger motions and augmented fingertips, the hand can perform diverse motions from pinching a nut🔩 to handling a pan🍳. Check the demos below🎥.

How do we make dexterous hands handle both power and precision tasks with ease? 🫳👌🫰 We introduce Power to Precision (💪➡️🎯), our new paper that optimizes both control and fingertip geometry to unlock robust manipulation from power grasp to fine-grained manipulations. With simplified finger motions and augmented fingertips, the hand can perform diverse motions from pinching a nut🔩 to handling a pan🍳. Check the demos below🎥.

Jianglong Ye

110,391 görüntüleme • 9 ay önce

How can robots acquire fine-grained manipulation skills? Introducing ACT: Action Chunking with Transformers 🤖 Key idea: Imitation, but predict actions in chunks instead of one at a time. Here are results with only ~15min of demonstrations, running on low-cost arms:

How can robots acquire fine-grained manipulation skills? Introducing ACT: Action Chunking with Transformers 🤖 Key idea: Imitation, but predict actions in chunks instead of one at a time. Here are results with only ~15min of demonstrations, running on low-cost arms:

Tony Zhao

247,710 görüntüleme • 3 yıl önce

Can VLMs enable robots to autonomously improve? In our new work we ran a fleet of robot arms to collect autonomous data with VLM-proposed tasks and showed that robots can keep getting better as they are deployed, without supervision: 🧵👇

Can VLMs enable robots to autonomously improve? In our new work we ran a fleet of robot arms to collect autonomous data with VLM-proposed tasks and showed that robots can keep getting better as they are deployed, without supervision: 🧵👇

Sergey Levine

19,449 görüntüleme • 2 yıl önce

Introducing 2️⃣ new AI systems for robotics: 🤖 ALOHA Unleashed to perform two-armed manipulation tasks 🦾 DemoStart to control a multi-fingered robotic hand They learned to tackle a range of actions requiring dexterity. Here's how. 🧵

Introducing 2️⃣ new AI systems for robotics: 🤖 ALOHA Unleashed to perform two-armed manipulation tasks 🦾 DemoStart to control a multi-fingered robotic hand They learned to tackle a range of actions requiring dexterity. Here's how. 🧵

Google DeepMind

297,295 görüntüleme • 1 yıl önce

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:

Google DeepMind

410,378 görüntüleme • 3 yıl önce

Excited to release RT-Affordance! We propose conditioning policies on visual affordance plans as an intermediate representation that allows us to learn new tasks without collecting any new robot trajectories. Website and paper: Here’s a short 🧵

Excited to release RT-Affordance! We propose conditioning policies on visual affordance plans as an intermediate representation that allows us to learn new tasks without collecting any new robot trajectories. Website and paper: Here’s a short 🧵

Soroush Nasiriany

27,495 görüntüleme • 1 yıl önce

Teaching robots to learn only from RGB human videos is hard! In Feel The Force (FTF), we teach robots to mimic the tactile feedback humans experience when handling objects. This allows for delicate, touch-sensitive tasks—like picking up a raw egg without breaking it. 🧵👇

Teaching robots to learn only from RGB human videos is hard! In Feel The Force (FTF), we teach robots to mimic the tactile feedback humans experience when handling objects. This allows for delicate, touch-sensitive tasks—like picking up a raw egg without breaking it. 🧵👇

Lerrel Pinto

70,328 görüntüleme • 1 yıl önce

Five years ago, OpenAI trained a 5-fingered humanoid hand to solve a Rubik’s Cube using RL and domain randomization, pushing the boundaries of sim-to-real for fine-motor tasks. It underscored the value of creating sufficiently complex simulated worlds in which robots can learn.

Five years ago, OpenAI trained a 5-fingered humanoid hand to solve a Rubik’s Cube using RL and domain randomization, pushing the boundaries of sim-to-real for fine-motor tasks. It underscored the value of creating sufficiently complex simulated worlds in which robots can learn.

The Humanoid Hub

66,761 görüntüleme • 1 yıl önce

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

Siyuan Huang

20,983 görüntüleme • 1 yıl önce

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 görüntüleme • 2 yıl önce

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Chen Wang

235,224 görüntüleme • 2 yıl önce

Robostral can now follow natural language instructions. It responds to voice commands and pointing. It is also getting better at fine-grained manipulation where precise force control matters. It generalizes to new objects and tasks not present in the training data.

Robostral can now follow natural language instructions. It responds to voice commands and pointing. It is also getting better at fine-grained manipulation where precise force control matters. It generalizes to new objects and tasks not present in the training data.

Olivier Duchenne

51,493 görüntüleme • 3 ay önce

Today we previewed Reinforcement Fine-Tuning, a new model customization technique that enables organizations to build expert models for specific, complex tasks in domains such as coding, scientific research, or finance.

Today we previewed Reinforcement Fine-Tuning, a new model customization technique that enables organizations to build expert models for specific, complex tasks in domains such as coding, scientific research, or finance.

OpenAI

1,072,900 görüntüleme • 1 yıl önce

Robotics just hit a dexterity milestone 🤯! Sharpa Robotics demonstrated autonomous dual-hand apple peeling using its new MoDE-VLA system. Using human-like dexterous hands with tactile sensing, the robot can feel contact and adjust its grip while rotating and peeling the apple. While the system achieved 30% success, the 73% peel completion rate shows robots are starting to master the fine, contact-rich manipulation needed for real world tasks.

Robotics just hit a dexterity milestone 🤯! Sharpa Robotics demonstrated autonomous dual-hand apple peeling using its new MoDE-VLA system. Using human-like dexterous hands with tactile sensing, the robot can feel contact and adjust its grip while rotating and peeling the apple. While the system achieved 30% success, the 73% peel completion rate shows robots are starting to master the fine, contact-rich manipulation needed for real world tasks.

SciTech Era

39,388 görüntüleme • 5 ay önce

Excited to share my final PhD project😀 We show how simple, yet elegant changes enable diffusion transformers to learn SOTA robotic policies on real robots. Our method improves performance by 20% across a wide range of highly dexterous tasks - like cutting sushi! 1/n

Excited to share my final PhD project😀 We show how simple, yet elegant changes enable diffusion transformers to learn SOTA robotic policies on real robots. Our method improves performance by 20% across a wide range of highly dexterous tasks - like cutting sushi! 1/n

Sudeep Dasari

20,536 görüntüleme • 1 yıl önce

Can we teach a robot its limits to do chores safely & correctly? 🧵 To help robots execute open-ended, multi-step tasks, MIT CSAIL researchers used vision models to see what’s near the machine & model its constraints. An LLM sketches up a plan that’s checked in a simulator to ensure it’s safe & realistic, potentially aiding household robots:

Can we teach a robot its limits to do chores safely & correctly? 🧵 To help robots execute open-ended, multi-step tasks, MIT CSAIL researchers used vision models to see what’s near the machine & model its constraints. An LLM sketches up a plan that’s checked in a simulator to ensure it’s safe & realistic, potentially aiding household robots:

MIT CSAIL

28,341 görüntüleme • 1 yıl önce

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

Google DeepMind

819,863 görüntüleme • 1 yıl önce

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Lucy Shi

122,827 görüntüleme • 2 yıl önce