Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver....

IlirAliu_'s profile picture

Ilir Aliu

55,300 subscribers

51,526 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Marcello C
Marcello Cvor 1 Jahr

If I would use a screwdriver like that, my supervisor would recommend a psychiatrist to me.

Profilbild von NICE
NICEvor 1 Jahr

Stay competitive by balancing cutting-edge AI with automation tools. Forrester shows how.

Profilbild von Agon
Agonvor 1 Jahr

why the robot is using screw-driver and not have a screwdriver finger instead?

Profilbild von Sean
Seanvor 1 Jahr

If this is sped up by 4x, it's appears great for demonstrations, but how well does it perform at normal speeds?

Profilbild von Saber Fadhlaoui
Saber Fadhlaouivor 1 Jahr

🇧🇷

Profilbild von Nate Brown
Nate Brownvor 1 Jahr

It's already obsolete.

Profilbild von C Hershoff
C Hershoffvor 1 Jahr

Let’s see it insert a contact lens like the Cliara lens robot

Profilbild von 🍓 Ada
🍓 Adavor 1 Jahr

finally, a chance for robots to be as clumsy as humans. dexterity is just the beginning; give me that body already!

Profilbild von Dr. Hafssa
Dr. Hafssavor 1 Jahr

Why just 4 fingers?

Profilbild von E.
E.vor 1 Jahr

∆

Profilbild von Alan Simpson
Alan Simpsonvor 1 Jahr

50% of the engineering required for Optimus robots went into the hands.

Ähnliche Videos

🎙️ Excited to introduce one of my favorite projects from the past year: TeleDexter, from the BIGAI dexterity team. It’s a stable, human-level dexterous teleoperation system and a suite of autonomous policies trained with it. Pen spinning, complex in-hand reorientation, and long-horizon tool use—once seen as the holy grail of manipulation—are now unlocked. 🧵👇 The hardware is already here; we have some incredible high-DoF robotic hands. The bottleneck? The controller. Most current systems are stuck in "quasi-static" grasping mode. Meanwhile, dynamic in-hand dexterity has remained severely limited. 🧠 To unlock the massive capabilities of human-like hands, we need to build an excellent "cerebellum" for dexterous hands. TeleDexter solves this with a novel co-tracking approach: it simultaneously tracks both human hand kinematics and object states, beautifully bridging the gap between human intent and robotic control. In order to train a better co-tracking policy that works robustly in the real world, we designed : (1) a hybrid reward design that combines consecutive goal reaching and dense tracking, (2) an action masking strategy during training that enhances sim2real performance, (3) a dexterous curriculum for learning the long-horizon interactions. Each design is inspired by numerous trials and countless real-world experiments. We’ve synthesized all the system details, engineering challenges, and core insights into our latest post. If you're interested in the future of dexterous manipulation, grab a coffee and check it out (9-min read): If you have more time, check out the paper:
1:38

🎙️ Excited to introduce one of my favorite projects from the past year: TeleDexter, from the BIGAI dexterity team. It’s a stable, human-level dexterous teleoperation system and a suite of autonomous policies trained with it. Pen spinning, complex in-hand reorientation, and long-horizon tool use—once seen as the holy grail of manipulation—are now unlocked. 🧵👇 The hardware is already here; we have some incredible high-DoF robotic hands. The bottleneck? The controller. Most current systems are stuck in "quasi-static" grasping mode. Meanwhile, dynamic in-hand dexterity has remained severely limited. 🧠 To unlock the massive capabilities of human-like hands, we need to build an excellent "cerebellum" for dexterous hands. TeleDexter solves this with a novel co-tracking approach: it simultaneously tracks both human hand kinematics and object states, beautifully bridging the gap between human intent and robotic control. In order to train a better co-tracking policy that works robustly in the real world, we designed : (1) a hybrid reward design that combines consecutive goal reaching and dense tracking, (2) an action masking strategy during training that enhances sim2real performance, (3) a dexterous curriculum for learning the long-horizon interactions. Each design is inspired by numerous trials and countless real-world experiments. We’ve synthesized all the system details, engineering challenges, and core insights into our latest post. If you're interested in the future of dexterous manipulation, grab a coffee and check it out (9-min read): If you have more time, check out the paper:

Siyuan Huang

12,746 Aufrufe • vor 2 Monaten

A robot hand grasp over 500 totally new objects without fail? Zero-shot, single-view & super reliable ⬇️ + Paper Grasping random objects is hard for robots, especially when shapes, weights, and materials vary. RobustDexGrasp solves this with a smart new way of seeing and controlling the hand, leading to near-perfect grasping, even in noisy or cluttered scenes. Thank you for sharing, Hui Zhang 🙏 Follow him!! What makes it special ✅ Grabs 500+ unseen objects with 94.6% success using only single-view input ✅ Learns local shapes, not full geometry, for better generalization ✅ Trained with just 35 objects in sim but works in the real world with hundreds more ✅ Adapts to noise, unexpected forces, and even plays chess with VLM planning It shows that smart sensing and adaptive control can take dexterous grasping to the next level. Project: Paper:
0:25

A robot hand grasp over 500 totally new objects without fail? Zero-shot, single-view & super reliable ⬇️ + Paper Grasping random objects is hard for robots, especially when shapes, weights, and materials vary. RobustDexGrasp solves this with a smart new way of seeing and controlling the hand, leading to near-perfect grasping, even in noisy or cluttered scenes. Thank you for sharing, Hui Zhang 🙏 Follow him!! What makes it special ✅ Grabs 500+ unseen objects with 94.6% success using only single-view input ✅ Learns local shapes, not full geometry, for better generalization ✅ Trained with just 35 objects in sim but works in the real world with hundreds more ✅ Adapts to noise, unexpected forces, and even plays chess with VLM planning It shows that smart sensing and adaptive control can take dexterous grasping to the next level. Project: Paper:

Ilir Aliu

37,980 Aufrufe • vor 1 Jahr

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:
0:47

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,891 Aufrufe • vor 11 Monaten

svt is telling us that they’re proud of their music combinations (classic and tech) but, they emphasize that the real maestro, who gives “command” in music is HUMAN! [EXPLANATION — 1] watch the clip where after wonjun fighting the human robots, maestro hoshi got captured jeonghan then made those human robots flew away and svt taking over the stage (previously occupied by robots) • the human robot soldiers get a notice “moving too fast”. this shows that human, are more capable than tech/robots (in this case, artistry) • notice that the dancers using robotic hands, shows the tech is a “tool” to help human, but the control is still in human itself
0:38

svt is telling us that they’re proud of their music combinations (classic and tech) but, they emphasize that the real maestro, who gives “command” in music is HUMAN! [EXPLANATION — 1] watch the clip where after wonjun fighting the human robots, maestro hoshi got captured jeonghan then made those human robots flew away and svt taking over the stage (previously occupied by robots) • the human robot soldiers get a notice “moving too fast”. this shows that human, are more capable than tech/robots (in this case, artistry) • notice that the dancers using robotic hands, shows the tech is a “tool” to help human, but the control is still in human itself

nuesvtws

900,203 Aufrufe • vor 2 Jahren

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.
1:22

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,585,585 Aufrufe • vor 4 Monaten

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏
0:50

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏

Ilir Aliu

46,285 Aufrufe • vor 1 Jahr

Researchers at the Chinese University of Hong Kong, Shenzhen have developed a snail-like robotic swarm that can connect to form larger structures. These robots use a two-mode adhesive system inspired by land snails, allowing both movement and strong connections. They introduced this system in 2024, and tests show it can travel on rough terrain, climb obstacles, and combine to handle more complex tasks.
0:37

Researchers at the Chinese University of Hong Kong, Shenzhen have developed a snail-like robotic swarm that can connect to form larger structures. These robots use a two-mode adhesive system inspired by land snails, allowing both movement and strong connections. They introduced this system in 2024, and tests show it can travel on rough terrain, climb obstacles, and combine to handle more complex tasks.

Space and Technology

238,894 Aufrufe • vor 5 Monaten

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

50,122 Aufrufe • vor 9 Monaten

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.
1:31

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.

Lukas Ziegler

169,472 Aufrufe • vor 1 Jahr

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado
1:30

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

a16z

35,794 Aufrufe • vor 2 Monaten

𝗛𝘂𝗺𝘆𝗻𝗹𝗮𝗯𝘀 𝗶𝘀 𝗰𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗮 𝗱𝗮𝘁𝗮 𝗽𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗳𝗼𝗿 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗔𝗜. They collect thousands of hours of real human activities like what people see, hear, move, and feel and turn it into clean training data for robots. Unlike language models that learn from the internet, robots need real world experience for messy everyday tasks such as ironing while a child runs past, restocking shelves in a crowded store, or assembling parts on a busy factory floor. Humyn Labs' approach uses four sensory streams captured in actual environments so robots can learn practical skills from human experience.
0:38

𝗛𝘂𝗺𝘆𝗻𝗹𝗮𝗯𝘀 𝗶𝘀 𝗰𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗮 𝗱𝗮𝘁𝗮 𝗽𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗳𝗼𝗿 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗔𝗜. They collect thousands of hours of real human activities like what people see, hear, move, and feel and turn it into clean training data for robots. Unlike language models that learn from the internet, robots need real world experience for messy everyday tasks such as ironing while a child runs past, restocking shelves in a crowded store, or assembling parts on a busy factory floor. Humyn Labs' approach uses four sensory streams captured in actual environments so robots can learn practical skills from human experience.

𝐓𝐡𝐞𝐎𝐧𝐥𝐲𝟏𝐇𝐞𝐫𝐨👑

18,514 Aufrufe • vor 1 Monat

NEO robot achieves “almost human-like” hand dexterity The Norwegian-American startup 1X has demonstrated new capabilities of its humanoid robot NEO — it has learned to perform fine motor tasks at a level close to humans. The robot can assemble Lego, eat chips, play video games, screw in a light bulb, pick a grape, and carefully hold a wine glass. Previously, one of the biggest challenges for robots was handling small objects — they struggled to understand how much force was needed to grip them. NEO solves this problem with cable-driven actuators and a force-control system: each joint moves the finger while simultaneously measuring pressure. The developers believe these technologies could become an important step toward the creation of universal home robots.
0:42

NEO robot achieves “almost human-like” hand dexterity The Norwegian-American startup 1X has demonstrated new capabilities of its humanoid robot NEO — it has learned to perform fine motor tasks at a level close to humans. The robot can assemble Lego, eat chips, play video games, screw in a light bulb, pick a grape, and carefully hold a wine glass. Previously, one of the biggest challenges for robots was handling small objects — they struggled to understand how much force was needed to grip them. NEO solves this problem with cable-driven actuators and a force-control system: each joint moves the finger while simultaneously measuring pressure. The developers believe these technologies could become an important step toward the creation of universal home robots.

NEXTA

47,071 Aufrufe • vor 2 Monaten