Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

𝗛𝘂𝗺𝘆𝗻𝗹𝗮𝗯𝘀 𝗶𝘀 𝗰𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗮 𝗱𝗮𝘁𝗮 𝗽𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗳𝗼𝗿 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗔𝗜. They collect thousands of hours of real human activities like what people see, hear, move, and feel and turn it into clean training data for robots. Unlike language models that learn from the internet, robots need real world experience for... show more

𝐓𝐡𝐞𝐎𝐧𝐥𝐲𝟏𝐇𝐞𝐫𝐨👑

1,493 subscribers

12,472 Aufrufe • vor 2 Tagen •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 Aufrufe • vor 1 Jahr

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,872 Aufrufe • vor 9 Monaten

Everyone is racing to build smarter robots. But here’s the question: Can a robot truly understand the physical world by only seeing it? Humans don’t learn with vision alone. We learn through sound, sight, motion, and touch… all working together to make sense of our environment. That’s what makes Humyn Labs so interesting. They’re building the Multi-Sensory Platform for Physical AI, creating one fused training signal instead of isolated datasets. And through KGeN Community’s verified global network, that data comes from real people performing real tasks in real environments. In this video, I explain why I think this approach could be one of the missing pieces for Physical AI’s next leap… and why I used the Infinity Gauntlet as the perfect analogy. 👇🎥 Watch, share your thoughts, and let me know if the comparison makes sense. Humyn Labs KGeN Community KGeN Foundation 🟩 #HumynLabs #KGeN #PhysicalAI #AI

Everyone is racing to build smarter robots. But here’s the question: Can a robot truly understand the physical world by only seeing it? Humans don’t learn with vision alone. We learn through sound, sight, motion, and touch… all working together to make sense of our environment. That’s what makes Humyn Labs so interesting. They’re building the Multi-Sensory Platform for Physical AI, creating one fused training signal instead of isolated datasets. And through KGeN Community’s verified global network, that data comes from real people performing real tasks in real environments. In this video, I explain why I think this approach could be one of the missing pieces for Physical AI’s next leap… and why I used the Infinity Gauntlet as the perfect analogy. 👇🎥 Watch, share your thoughts, and let me know if the comparison makes sense. Humyn Labs KGeN Community KGeN Foundation 🟩 #HumynLabs #KGeN #PhysicalAI #AI

Bahdfella™ ❤️‍🔥

28,381 Aufrufe • vor 28 Tagen

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Sawyer Merritt

250,956 Aufrufe • vor 6 Monaten

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine

61,187 Aufrufe • vor 4 Monaten

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Teslaconomics

42,563 Aufrufe • vor 6 Monaten

Robots can already fold laundry, make espresso, clean kitchens, and assemble things. The harder problem is getting them to do those tasks reliably, for long periods of time, without a human babysitting them. At Startup School 2026, Physical Intelligence cofounder Chelsea Finn explains what it takes to build general-purpose robots that work in the real world. She shares how reinforcement learning pushed robot throughput up 2x, how their systems can run autonomously for hours, and why she believes robotics is entering its GPT era: moving from specialized models toward general-purpose systems that can work across tasks, robots, and environments. 00:00 — The State of Physical Intelligence 01:23 — What It Takes to Make Robots Useful 05:11 — The Reliability Problem 07:43 — Reinforcement Learning for Robotics 09:35 — Learning From Failures 12:43 — Training Robots to Improve Themselves 14:21 — Can a Robot Work for 13 Hours Straight? 17:36 — Why Robots Need Memory 21:22 — Building a General-Purpose Robot 25:02 — From Fine-Tuning to Out-of-the-Box Models 27:35 — Training on All the Data 30:20 — One Model That Beats the Specialists 31:21 — Compositional Generalization 37:49 — The GPT Era of Robotics 39:49 — Q&A

Robots can already fold laundry, make espresso, clean kitchens, and assemble things. The harder problem is getting them to do those tasks reliably, for long periods of time, without a human babysitting them. At Startup School 2026, Physical Intelligence cofounder Chelsea Finn explains what it takes to build general-purpose robots that work in the real world. She shares how reinforcement learning pushed robot throughput up 2x, how their systems can run autonomously for hours, and why she believes robotics is entering its GPT era: moving from specialized models toward general-purpose systems that can work across tasks, robots, and environments. 00:00 — The State of Physical Intelligence 01:23 — What It Takes to Make Robots Useful 05:11 — The Reliability Problem 07:43 — Reinforcement Learning for Robotics 09:35 — Learning From Failures 12:43 — Training Robots to Improve Themselves 14:21 — Can a Robot Work for 13 Hours Straight? 17:36 — Why Robots Need Memory 21:22 — Building a General-Purpose Robot 25:02 — From Fine-Tuning to Out-of-the-Box Models 27:35 — Training on All the Data 30:20 — One Model That Beats the Specialists 31:21 — Compositional Generalization 37:49 — The GPT Era of Robotics 39:49 — Q&A

Y Combinator

105,935 Aufrufe • vor 14 Tagen

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

a16z

35,480 Aufrufe • vor 28 Tagen

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

50,090 Aufrufe • vor 8 Monaten

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:

Ilir Aliu

51,490 Aufrufe • vor 1 Jahr

Amazon is training humanoids to move boxes. Makes sense! OmniRetarget is a data generation engine that enables complex loco-manipulation for humanoids. It uses offline retargeting from human MoCap datasets and augments data from single demos to produce 8 hours of trajectories that train RL policies in simulation, transferred zero-shot to real robots. Amazon FAR is the primary affiliation for the paper:

Amazon is training humanoids to move boxes. Makes sense! OmniRetarget is a data generation engine that enables complex loco-manipulation for humanoids. It uses offline retargeting from human MoCap datasets and augments data from single demos to produce 8 hours of trajectories that train RL policies in simulation, transferred zero-shot to real robots. Amazon FAR is the primary affiliation for the paper:

The Humanoid Hub

106,447 Aufrufe • vor 10 Monaten

1/7 Physical AI has a data problem. More compute alone won’t teach a robot how to move, grasp, react, or recover from mistakes. LLMs had the entire internet to learn from. Robots need something different: huge amounts of real action data. That’s where Axis Robotics gets interesting. $AXIS is building a data engine for Physical AI, turning robot training into something people can contribute to directly through a browser. The loop is straightforward: Generate → Collect → Process → Deploy → Correct → Repeat. What looks simple on the surface could become something much bigger underneath: a scalable way to produce the training data robots need to become genuinely useful in the physical world. The real unlock for Physical AI might not be another bigger model. It might be better data.

1/7 Physical AI has a data problem. More compute alone won’t teach a robot how to move, grasp, react, or recover from mistakes. LLMs had the entire internet to learn from. Robots need something different: huge amounts of real action data. That’s where Axis Robotics gets interesting. $AXIS is building a data engine for Physical AI, turning robot training into something people can contribute to directly through a browser. The loop is straightforward: Generate → Collect → Process → Deploy → Correct → Repeat. What looks simple on the surface could become something much bigger underneath: a scalable way to produce the training data robots need to become genuinely useful in the physical world. The real unlock for Physical AI might not be another bigger model. It might be better data.

_victorad

18,832 Aufrufe • vor 1 Tag

A simple idea. Let robots collect the data that current foundation models are missing. A robot that gets better by doing real work in the real world. For two weeks in the Stanford East Asia Library, Scanford scanned shelves, helped librarians, and improved the vision language model it depends on. The idea is very simple: Robots do useful work. They gather the real world data foundation models never see online. They fine tune their own model They go back out stronger A full loop. What they found in deployment: ✅ 2103 shelves scanned with multilingual, faded, occluded book spines ✅ 18.7 hours of librarian time saved ✅ Book ID accuracy jumped from 32.0 percent to 71.8 percent ✅ English OCR improved from 24.8 percent to 46.6 percent ✅ Chinese OCR improved from 30.8 percent to 38.0 percent The most interesting part is the shift. Robots do not only consume foundation models. They create the data these models are missing. A clean robot powered data flywheel. Work. Collect. Fine tune. Repeat. Thanks for sharing, Jenn Grannen! If you want the full write up: 📍Website: Paper: —- Weekly robotics and AI insights. Subscribe free:

A simple idea. Let robots collect the data that current foundation models are missing. A robot that gets better by doing real work in the real world. For two weeks in the Stanford East Asia Library, Scanford scanned shelves, helped librarians, and improved the vision language model it depends on. The idea is very simple: Robots do useful work. They gather the real world data foundation models never see online. They fine tune their own model They go back out stronger A full loop. What they found in deployment: ✅ 2103 shelves scanned with multilingual, faded, occluded book spines ✅ 18.7 hours of librarian time saved ✅ Book ID accuracy jumped from 32.0 percent to 71.8 percent ✅ English OCR improved from 24.8 percent to 46.6 percent ✅ Chinese OCR improved from 30.8 percent to 38.0 percent The most interesting part is the shift. Robots do not only consume foundation models. They create the data these models are missing. A clean robot powered data flywheel. Work. Collect. Fine tune. Repeat. Thanks for sharing, Jenn Grannen! If you want the full write up: 📍Website: Paper: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

44,660 Aufrufe • vor 9 Monaten

Elon Musk on how AGI might actually emerge: “We’re not actually specifically focused on AGI. I’m simply saying that AGI seems likely to be an emergent property of what we’re doing” The reason is the massive real-world data Tesla is generating from autonomous cars and humanoid robots This is not internet data. This is real-world interaction driving on messy roads, navigating unpredictable environments, and interacting with people With tens or hundreds of millions of autonomous vehicles and potentially even more humanoids, the amount of real-world video and data being processed will be enormous From this, cars will become far better than human drivers, and humanoid robots will become increasingly indistinguishable from humans AGI might not come from a lab trying to build it directly. It may simply emerge from building useful robots at massive scale

Elon Musk on how AGI might actually emerge: “We’re not actually specifically focused on AGI. I’m simply saying that AGI seems likely to be an emergent property of what we’re doing” The reason is the massive real-world data Tesla is generating from autonomous cars and humanoid robots This is not internet data. This is real-world interaction driving on messy roads, navigating unpredictable environments, and interacting with people With tens or hundreds of millions of autonomous vehicles and potentially even more humanoids, the amount of real-world video and data being processed will be enormous From this, cars will become far better than human drivers, and humanoid robots will become increasingly indistinguishable from humans AGI might not come from a lab trying to build it directly. It may simply emerge from building useful robots at massive scale

X Freeze

21,256 Aufrufe • vor 2 Monaten

😱 Humanoid robots just performed surgery on live animals for the first time ever At UC San Diego, two humanoid robots successfully removed gallbladders from pigs. In one procedure, a robot assisted a human surgeon. In the second, two robots operated together at the table. Important note: The robots didn’t make any independent decisions — they were fully teleoperated by real doctors. But unlike bulky traditional surgical robots, these humanoids weigh just ~30 kg, move freely around the OR, and use the exact same instruments as human surgeons. This is a major milestone. The next step? Autonomy. Once achieved, these robots could become true surgical assistants, handling routine parts of procedures without constant human control. Medicine is entering a wild new era. What do you think — exciting or slightly terrifying?

😱 Humanoid robots just performed surgery on live animals for the first time ever At UC San Diego, two humanoid robots successfully removed gallbladders from pigs. In one procedure, a robot assisted a human surgeon. In the second, two robots operated together at the table. Important note: The robots didn’t make any independent decisions — they were fully teleoperated by real doctors. But unlike bulky traditional surgical robots, these humanoids weigh just ~30 kg, move freely around the OR, and use the exact same instruments as human surgeons. This is a major milestone. The next step? Autonomy. Once achieved, these robots could become true surgical assistants, handling routine parts of procedures without constant human control. Medicine is entering a wild new era. What do you think — exciting or slightly terrifying?

NEXTA

52,220 Aufrufe • vor 1 Monat

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

15,092 Aufrufe • vor 9 Monaten

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?

Lukas Ziegler

61,610 Aufrufe • vor 1 Jahr