Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

Chelsea Finn

99,275 subscribers

264,480 Aufrufe • vor 3 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Chelsea Finn

Chelsea Finnvor 3 Jahren

First, the hardware: We use simple puppeteering. Just copy the joint angles from the leader to the follower robot. No tactile or force feedback. The manufacturer (@trossenrobotics) didn’t know these tasks were possible.

Profilbild von Chelsea Finn

Chelsea Finnvor 3 Jahren

Can the robot do all this on its own? We train the robot to predict actions in chunks, rather than one at a time. Recipe: action chunking + transformers + only 50 demonstrations The robot *autonomously* completes fine manipulation skills. ⬇️

Profilbild von Chelsea Finn

Chelsea Finnvor 3 Jahren

But wait, there’s more! Website & Code: Paper: You can buy the full kit directly from Trossen: Led by the amazing @tonyzzhao, with @svlevine, @Vikashplus

Profilbild von Deepak Pathak

Deepak Pathakvor 3 Jahren

Great to see it being finally released. Looks very cool. Congrats @chelseabfinn and @tonyzzhao !!

Profilbild von Danijar Hafner

Danijar Hafnervor 3 Jahren

This looks super cool, congrats!! To what extent do the policies generalize (e.g. object variations, positions)?

Profilbild von Chelsea Finn

Chelsea Finnvor 3 Jahren

Thanks @danijarh! The demos include ~15cm of object position variation, and the policies generalize to that degree. Need more & wider demo data for more generalization. We also see some generalization to distractors & disturbances.

Profilbild von Co Builder

Co Buildervor 3 Jahren

This is mind blowing!! 🤩! Congrats!!

Profilbild von Tran Van Phu

Tran Van Phuvor 3 Jahren

@LukeW I'm having the fastest hosting and utility design. Who needs can contact:

Profilbild von Snm_Str

Snm_Strvor 3 Jahren

@futuristufuk

Profilbild von Multimodal

Multimodalvor 3 Jahren

Impressive!

Ähnliche Videos

How do we make dexterous hands handle both power and precision tasks with ease? 🫳👌🫰 We introduce Power to Precision (💪➡️🎯), our new paper that optimizes both control and fingertip geometry to unlock robust manipulation from power grasp to fine-grained manipulations. With simplified finger motions and augmented fingertips, the hand can perform diverse motions from pinching a nut🔩 to handling a pan🍳. Check the demos below🎥.

How do we make dexterous hands handle both power and precision tasks with ease? 🫳👌🫰 We introduce Power to Precision (💪➡️🎯), our new paper that optimizes both control and fingertip geometry to unlock robust manipulation from power grasp to fine-grained manipulations. With simplified finger motions and augmented fingertips, the hand can perform diverse motions from pinching a nut🔩 to handling a pan🍳. Check the demos below🎥.

Jianglong Ye

110,220 Aufrufe • vor 9 Monaten

How can robots acquire fine-grained manipulation skills? Introducing ACT: Action Chunking with Transformers 🤖 Key idea: Imitation, but predict actions in chunks instead of one at a time. Here are results with only ~15min of demonstrations, running on low-cost arms:

How can robots acquire fine-grained manipulation skills? Introducing ACT: Action Chunking with Transformers 🤖 Key idea: Imitation, but predict actions in chunks instead of one at a time. Here are results with only ~15min of demonstrations, running on low-cost arms:

Tony Zhao

247,710 Aufrufe • vor 3 Jahren

Can VLMs enable robots to autonomously improve? In our new work we ran a fleet of robot arms to collect autonomous data with VLM-proposed tasks and showed that robots can keep getting better as they are deployed, without supervision: 🧵👇

Can VLMs enable robots to autonomously improve? In our new work we ran a fleet of robot arms to collect autonomous data with VLM-proposed tasks and showed that robots can keep getting better as they are deployed, without supervision: 🧵👇

Sergey Levine

19,449 Aufrufe • vor 2 Jahren

Introducing 2️⃣ new AI systems for robotics: 🤖 ALOHA Unleashed to perform two-armed manipulation tasks 🦾 DemoStart to control a multi-fingered robotic hand They learned to tackle a range of actions requiring dexterity. Here's how. 🧵

Introducing 2️⃣ new AI systems for robotics: 🤖 ALOHA Unleashed to perform two-armed manipulation tasks 🦾 DemoStart to control a multi-fingered robotic hand They learned to tackle a range of actions requiring dexterity. Here's how. 🧵

Google DeepMind

297,295 Aufrufe • vor 1 Jahr

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:

Google DeepMind

410,378 Aufrufe • vor 3 Jahren

Excited to release RT-Affordance! We propose conditioning policies on visual affordance plans as an intermediate representation that allows us to learn new tasks without collecting any new robot trajectories. Website and paper: Here’s a short 🧵

Excited to release RT-Affordance! We propose conditioning policies on visual affordance plans as an intermediate representation that allows us to learn new tasks without collecting any new robot trajectories. Website and paper: Here’s a short 🧵

Soroush Nasiriany

27,495 Aufrufe • vor 1 Jahr

Teaching robots to learn only from RGB human videos is hard! In Feel The Force (FTF), we teach robots to mimic the tactile feedback humans experience when handling objects. This allows for delicate, touch-sensitive tasks—like picking up a raw egg without breaking it. 🧵👇

Teaching robots to learn only from RGB human videos is hard! In Feel The Force (FTF), we teach robots to mimic the tactile feedback humans experience when handling objects. This allows for delicate, touch-sensitive tasks—like picking up a raw egg without breaking it. 🧵👇

Lerrel Pinto

70,328 Aufrufe • vor 1 Jahr

Five years ago, OpenAI trained a 5-fingered humanoid hand to solve a Rubik’s Cube using RL and domain randomization, pushing the boundaries of sim-to-real for fine-motor tasks. It underscored the value of creating sufficiently complex simulated worlds in which robots can learn.

Five years ago, OpenAI trained a 5-fingered humanoid hand to solve a Rubik’s Cube using RL and domain randomization, pushing the boundaries of sim-to-real for fine-motor tasks. It underscored the value of creating sufficiently complex simulated worlds in which robots can learn.

The Humanoid Hub

66,761 Aufrufe • vor 1 Jahr

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

📢📢📢 Excited to release ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR25). 🤏🤙✌️With ManipTrans, we can transfer dexterous manipulation skills into robotic hands in simulation and deploy them on a real robot, using a residual policy learned for dex manipulation. 🤖🤖🤖The video below illustrates how the MoCap data can be transferred to Inspire, Shadow, Xhand, Allegro, and Mano. With ManipTrans, we can scale up dex manip data greatly with minimal effort. For more details, please check our -webpage: -code: -huggingface:

Siyuan Huang

20,975 Aufrufe • vor 1 Jahr

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 Aufrufe • vor 2 Jahren

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced

Chen Wang

235,224 Aufrufe • vor 2 Jahren

Robostral can now follow natural language instructions. It responds to voice commands and pointing. It is also getting better at fine-grained manipulation where precise force control matters. It generalizes to new objects and tasks not present in the training data.

Robostral can now follow natural language instructions. It responds to voice commands and pointing. It is also getting better at fine-grained manipulation where precise force control matters. It generalizes to new objects and tasks not present in the training data.

Olivier Duchenne

51,493 Aufrufe • vor 2 Monaten

Today we previewed Reinforcement Fine-Tuning, a new model customization technique that enables organizations to build expert models for specific, complex tasks in domains such as coding, scientific research, or finance.

Today we previewed Reinforcement Fine-Tuning, a new model customization technique that enables organizations to build expert models for specific, complex tasks in domains such as coding, scientific research, or finance.

OpenAI

1,072,863 Aufrufe • vor 1 Jahr

Excited to share my final PhD project😀 We show how simple, yet elegant changes enable diffusion transformers to learn SOTA robotic policies on real robots. Our method improves performance by 20% across a wide range of highly dexterous tasks - like cutting sushi! 1/n

Excited to share my final PhD project😀 We show how simple, yet elegant changes enable diffusion transformers to learn SOTA robotic policies on real robots. Our method improves performance by 20% across a wide range of highly dexterous tasks - like cutting sushi! 1/n

Sudeep Dasari

20,536 Aufrufe • vor 1 Jahr

Robotics just hit a dexterity milestone 🤯! Sharpa Robotics demonstrated autonomous dual-hand apple peeling using its new MoDE-VLA system. Using human-like dexterous hands with tactile sensing, the robot can feel contact and adjust its grip while rotating and peeling the apple. While the system achieved 30% success, the 73% peel completion rate shows robots are starting to master the fine, contact-rich manipulation needed for real world tasks.

Robotics just hit a dexterity milestone 🤯! Sharpa Robotics demonstrated autonomous dual-hand apple peeling using its new MoDE-VLA system. Using human-like dexterous hands with tactile sensing, the robot can feel contact and adjust its grip while rotating and peeling the apple. While the system achieved 30% success, the 73% peel completion rate shows robots are starting to master the fine, contact-rich manipulation needed for real world tasks.

SciTech Era

39,388 Aufrufe • vor 5 Monaten

Can we teach a robot its limits to do chores safely & correctly? 🧵 To help robots execute open-ended, multi-step tasks, MIT CSAIL researchers used vision models to see what’s near the machine & model its constraints. An LLM sketches up a plan that’s checked in a simulator to ensure it’s safe & realistic, potentially aiding household robots:

Can we teach a robot its limits to do chores safely & correctly? 🧵 To help robots execute open-ended, multi-step tasks, MIT CSAIL researchers used vision models to see what’s near the machine & model its constraints. An LLM sketches up a plan that’s checked in a simulator to ensure it’s safe & realistic, potentially aiding household robots:

MIT CSAIL

28,341 Aufrufe • vor 1 Jahr

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

Google DeepMind

819,819 Aufrufe • vor 1 Jahr

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Lucy Shi

122,827 Aufrufe • vor 2 Jahren