Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

How to chain multiple dexterous skills to tackle complex long-horizon manipulation tasks? Imagine retrieving a LEGO block from a pile, rotating it in-hand, and inserting it at the desired location to build a structure. Introducing our new work - Sequential Dexterity 🧵👇

Chen Wang

3,414 subscribers

159,809 Aufrufe • vor 2 Jahren •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Chen Wang

Chen Wangvor 2 Jahren

The core of the system is a learning-based transition feasibility function that progressively finetunes the sub-policies (learned with RL) for enhancing chaining success, which can also be used during skill selection for re-planning from failures and bypassing redundant stages.

Profilbild von Chen Wang

Chen Wangvor 2 Jahren

Despite being trained only in simulation with a few task objects, our system demonstrates generalization capability to novel object shapes and is able to zero-shot transfer to a real-world robot equipped with a dexterous hand.

Profilbild von Chen Wang

Chen Wangvor 2 Jahren

We hope Sequential Dexterity paves the path for future research on long-horizon dexterous manipulation. Feel free to check out our code! Website & Paper: Code: Work done w/ Yuanpei Chen, @drfeifei, and Karen Liu at @StanfordAILab.

Profilbild von Karol Hausman

Karol Hausmanvor 2 Jahren

Congrats, great work!

Profilbild von Chen Wang

Chen Wangvor 2 Jahren

Thanks Karol!

Profilbild von Deepak Pathak

Deepak Pathakvor 2 Jahren

very cool task and results, congrats!!

Profilbild von Chen Wang

Chen Wangvor 2 Jahren

Thanks Deepak! Very interested in LEAP hand and functional grasp

Profilbild von Lucy Shi

Lucy Shivor 2 Jahren

super cool work, @chenwang_j!

Profilbild von Chen Wang

Chen Wangvor 2 Jahren

Thanks Lucy!

Profilbild von Kevin Zakka

Kevin Zakkavor 2 Jahren

Very inspiring, congrats!

Profilbild von Chen Wang

Chen Wangvor 2 Jahren

Thanks Kevin! huge fan of RoboPianist!

Ähnliche Videos

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

How to learn dexterous manipulation for any robot hand from a single human demonstration? Check out DexMachina, our new RL algorithm that learns long-horizon, bimanual dexterous policies for a variety of dexterous hands, articulated objects, and complex motions.

Mandi Zhao

121,265 Aufrufe • vor 1 Jahr

Introducing 2️⃣ new AI systems for robotics: 🤖 ALOHA Unleashed to perform two-armed manipulation tasks 🦾 DemoStart to control a multi-fingered robotic hand They learned to tackle a range of actions requiring dexterity. Here's how. 🧵

Introducing 2️⃣ new AI systems for robotics: 🤖 ALOHA Unleashed to perform two-armed manipulation tasks 🦾 DemoStart to control a multi-fingered robotic hand They learned to tackle a range of actions requiring dexterity. Here's how. 🧵

Google DeepMind

297,118 Aufrufe • vor 1 Jahr

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

Xiao Ma

46,323 Aufrufe • vor 1 Jahr

A few weeks ago, we shared our progress on articulated objects and long-horizon tasks. Here are two representative examples: - We've been steadily expanding our asset library to cover more articulated objects. Articulated objects have always been a challenging asset class to handle in simulation. Interacting with them requires robots to master atomic skills such as pushing, pulling, opening, and closing, and to understand part structure, interaction constraints, and how the object moves. - Long-horizon tasks can now be generated at scale. Long-horizon tasks are the other hard category: they require chaining multiple sub-goals in sequence. A failure early in the task can cascade and make the rest unrecoverable. Axis is scaling along three dimensions at once: data volume, data quality, and task difficulty.

A few weeks ago, we shared our progress on articulated objects and long-horizon tasks. Here are two representative examples: - We've been steadily expanding our asset library to cover more articulated objects. Articulated objects have always been a challenging asset class to handle in simulation. Interacting with them requires robots to master atomic skills such as pushing, pulling, opening, and closing, and to understand part structure, interaction constraints, and how the object moves. - Long-horizon tasks can now be generated at scale. Long-horizon tasks are the other hard category: they require chaining multiple sub-goals in sequence. A failure early in the task can cascade and make the rest unrecoverable. Axis is scaling along three dimensions at once: data volume, data quality, and task difficulty.

Axis Robotics

13,700 Aufrufe • vor 1 Monat

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Lucy Shi

122,827 Aufrufe • vor 2 Jahren

Tip: Use Leo's Multi-Tab Context to draw information from several tabs at once. 📚 It's a great way to complete complex tasks with our AI assistant. Here, we're using it to build a shopping list based on multiple recipes.

Tip: Use Leo's Multi-Tab Context to draw information from several tabs at once. 📚 It's a great way to complete complex tasks with our AI assistant. Here, we're using it to build a shopping list based on multiple recipes.

Brave

65,096 Aufrufe • vor 1 Jahr

At the Film Culture Carnival in Ningbo, a robot steps onto a film set, taking on tasks from camera work to visual effects support. Watch the video to see it in action. [Video/Hi New Horizon]

At the Film Culture Carnival in Ningbo, a robot steps onto a film set, taking on tasks from camera work to visual effects support. Watch the video to see it in action. [Video/Hi New Horizon]

Insight Ningbo

99,590 Aufrufe • vor 3 Monaten

Thrilled to share one of my favorite works this year: DexNDM! We bridge the Sim2Real gap for dexterous in-hand rotation, achieving a true "0-to-1" advancement. The key? DexNDM learns from biased, real-world data without needing any successful demonstrations. Now a general-purpose dexterous hand can stably rotate large books, long rods, & complex objects around any axis, from any wrist pose. This powerful primitive enables complex, long-horizon tasks like teleoperated screwing and furniture assembly. 📄 Paper: 🌐 Project Page:

Thrilled to share one of my favorite works this year: DexNDM! We bridge the Sim2Real gap for dexterous in-hand rotation, achieving a true "0-to-1" advancement. The key? DexNDM learns from biased, real-world data without needing any successful demonstrations. Now a general-purpose dexterous hand can stably rotate large books, long rods, & complex objects around any axis, from any wrist pose. This powerful primitive enables complex, long-horizon tasks like teleoperated screwing and furniture assembly. 📄 Paper: 🌐 Project Page:

Li Yi

38,751 Aufrufe • vor 9 Monaten

ViTacFormer is a unified visuo-tactile framework for dexterous robot manipulation. It fuses high-res visual+tactile data using cross-attention and predicts future tactile signals via an autoregressive head, enabling multi-fingered hands to perform precise, long-horizon tasks.

ViTacFormer is a unified visuo-tactile framework for dexterous robot manipulation. It fuses high-res visual+tactile data using cross-attention and predicts future tactile signals via an autoregressive head, enabling multi-fingered hands to perform precise, long-horizon tasks.

The Humanoid Hub

10,816 Aufrufe • vor 1 Jahr

Imitation learning has a data scarcity problem. Introducing EgoDex from Apple, the largest and most diverse dataset of dexterous human manipulation to date — 829 hours of egocentric video + paired 3D hand poses across 194 tasks. Now on arxiv: (1/4)

Imitation learning has a data scarcity problem. Introducing EgoDex from Apple, the largest and most diverse dataset of dexterous human manipulation to date — 829 hours of egocentric video + paired 3D hand poses across 194 tasks. Now on arxiv: (1/4)

Ryan Hoque

114,164 Aufrufe • vor 1 Jahr

Introducing GPT-5.5 A new class of intelligence for real work and powering agents, built to understand complex goals, use tools, check its work, and carry more tasks through to completion. It marks a new way of getting computer work done. Now available in ChatGPT and Codex.

Introducing GPT-5.5 A new class of intelligence for real work and powering agents, built to understand complex goals, use tools, check its work, and carry more tasks through to completion. It marks a new way of getting computer work done. Now available in ChatGPT and Codex.

OpenAI

13,194,582 Aufrufe • vor 3 Monaten

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:

Introducing RoboCat, a new AI model designed to operate multiple robots. 🤖 It learns to solve new tasks on different robotic arms with as few as 100 demonstrations - and improves skills from self-generated training data. Find out more:

Google DeepMind

410,368 Aufrufe • vor 3 Jahren

Learn how to build a Gemini 3 Deep Agents with this video from LangChain. These agents can reason through complex, long horizon tasks by: > Breaking goals into actionable steps > Delegating specific work for specialized models > Leveraging file systems and code execution

Learn how to build a Gemini 3 Deep Agents with this video from LangChain. These agents can reason through complex, long horizon tasks by: > Breaking goals into actionable steps > Delegating specific work for specialized models > Leveraging file systems and code execution

Google AI Developers

19,484 Aufrufe • vor 8 Monaten

Can robotic hands really outperform human dexterity? Meet the Linker Hand L30 from LinkerBot. This tendon-driven dexterous hand packs high precision and compact size, making it a serious contender for complex manipulation tasks: ► 22 Degrees of Freedom utilizing a sophisticated tendon drive system. ► Ultra-fine positioning control with ±0.20mm repeatable accuracy. ► Rapid response speed, achieving up to 450°/S for core movements. ► Adaptive grasping enabled by high-sensitivity tactile sensors built into the fingertips. ► Compact dimensions (85mm average width, 179mm length) despite its complex architecture.

Can robotic hands really outperform human dexterity? Meet the Linker Hand L30 from LinkerBot. This tendon-driven dexterous hand packs high precision and compact size, making it a serious contender for complex manipulation tasks: ► 22 Degrees of Freedom utilizing a sophisticated tendon drive system. ► Ultra-fine positioning control with ±0.20mm repeatable accuracy. ► Rapid response speed, achieving up to 450°/S for core movements. ► Adaptive grasping enabled by high-sensitivity tactile sensors built into the fingertips. ► Compact dimensions (85mm average width, 179mm length) despite its complex architecture.

RoboHub🤖

22,556 Aufrufe • vor 10 Monaten

First look at our collaboration with Audi on bringing AI-driven robotics into industrial production. Our end-to-end pixel-to-action model, running on our bi-manual platform, is capable of performing a complex, dexterous and long-horizon insertion task.

First look at our collaboration with Audi on bringing AI-driven robotics into industrial production. Our end-to-end pixel-to-action model, running on our bi-manual platform, is capable of performing a complex, dexterous and long-horizon insertion task.

mimic

65,860 Aufrufe • vor 5 Monaten

wow Homemade robotic hand & wrist doing actual stuff 👀 It features a high-fidelity robotic hand or bionic prosthetic designed for advanced dexterous manipulation. And engineered to mimic natural human motion, capable of precise tasks such as holding a pen or manipulating tools. It uses 3D-printed components and electronic actuators to mimic human hand movements.

wow Homemade robotic hand & wrist doing actual stuff 👀 It features a high-fidelity robotic hand or bionic prosthetic designed for advanced dexterous manipulation. And engineered to mimic natural human motion, capable of precise tasks such as holding a pen or manipulating tools. It uses 3D-printed components and electronic actuators to mimic human hand movements.

SciTech Era

14,975 Aufrufe • vor 3 Monaten

Can a dexterous hand still function after being run over by a 2-ton car? The dexterous hand is the most expensive component of a humanoid robot, and if it breaks, the task is interrupted. Hangzhou BrainCo conducted a rigorous test on its latest dexterous hand, Revo2: it was run over by a nearly 2-ton Volvo car, dropped from a height of 5m, dropped 30m, and subjected to forceful impacts from a Unitree G1 robot. It still managed to maintain a stable grip and continue operating. Revo2 is BrainCo's next-generation dexterous/bionic hand, released last year. yes, a human-robot hybrid hand. It's very light, only 383g, 20kg payload,capable of precision manipulation down to 0.1mm (such as threading a needle), and possesses multi-dimensional tactile perception (hardness, texture, direction, proximity, etc.). This test also demonstrates that after achieving breakthroughs in performance, a dexterous hand also needs to be extremely durable and able to function under extreme environmental pressures.

Can a dexterous hand still function after being run over by a 2-ton car? The dexterous hand is the most expensive component of a humanoid robot, and if it breaks, the task is interrupted. Hangzhou BrainCo conducted a rigorous test on its latest dexterous hand, Revo2: it was run over by a nearly 2-ton Volvo car, dropped from a height of 5m, dropped 30m, and subjected to forceful impacts from a Unitree G1 robot. It still managed to maintain a stable grip and continue operating. Revo2 is BrainCo's next-generation dexterous/bionic hand, released last year. yes, a human-robot hybrid hand. It's very light, only 383g, 20kg payload,capable of precision manipulation down to 0.1mm (such as threading a needle), and possesses multi-dimensional tactile perception (hardness, texture, direction, proximity, etc.). This test also demonstrates that after achieving breakthroughs in performance, a dexterous hand also needs to be extremely durable and able to function under extreme environmental pressures.

CyberRobo

45,434 Aufrufe • vor 5 Monaten

We’re excited to launch the Codex app, a command center for building with agents. It gives you a focused space to manage multiple agents at once, run work in parallel, and collaborate with agents over long-running tasks.

We’re excited to launch the Codex app, a command center for building with agents. It gives you a focused space to manage multiple agents at once, run work in parallel, and collaborate with agents over long-running tasks.

OpenAI Developers

1,305,114 Aufrufe • vor 6 Monaten