Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Robots Using Their Feet to Manipulate Objects ETH Zurich researchers have advanced "pedipulation," where legged robots use their feet for manipulation instead of robotic arms. Their new approach trains robots to navigate and manipulate objects while avoiding static and dynamic obstacles. Tested on the ANYmal quadruped, the method enables... show more

Circuit

3,936 subscribers

11,802 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Robot dog learned dexterity! 🤯 Robots usually struggle to move and handle objects at the same time, a challenge called loco-manipulation. Researchers at the RAI Institute have developed a new system called ReLIC that makes this easier. Instead of giving robots fixed roles for arms and legs, ReLIC lets them switch roles on the fly, for example, using a leg to help lift something while balancing on the others. This approach is more natural, similar to how humans use different body parts together without thinking about it. ReLIC was tested on a four-legged robot, which successfully performed tasks like moving boxes, opening drawers, and placing items into containers.

Robot dog learned dexterity! 🤯 Robots usually struggle to move and handle objects at the same time, a challenge called loco-manipulation. Researchers at the RAI Institute have developed a new system called ReLIC that makes this easier. Instead of giving robots fixed roles for arms and legs, ReLIC lets them switch roles on the fly, for example, using a leg to help lift something while balancing on the others. This approach is more natural, similar to how humans use different body parts together without thinking about it. ReLIC was tested on a four-legged robot, which successfully performed tasks like moving boxes, opening drawers, and placing items into containers.

Lukas Ziegler

19,743 views • 10 months ago

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

RoboPapers

44,803 views • 8 months ago

This robodog aims to remove humans from hazardous work environments The Robot Teleoperativo 2 project out of the Dynamic Legged Systems lab has developed an advanced teleoperation system designed to operate in high-risk environments, minimizing dangers to human workers. By integrating cutting-edge technologies in tele-locomotion, tele-manipulation, and remote human-robot interaction, the system enables precise, dual-arm control for complex tasks in hazardous conditions.

This robodog aims to remove humans from hazardous work environments The Robot Teleoperativo 2 project out of the Dynamic Legged Systems lab has developed an advanced teleoperation system designed to operate in high-risk environments, minimizing dangers to human workers. By integrating cutting-edge technologies in tele-locomotion, tele-manipulation, and remote human-robot interaction, the system enables precise, dual-arm control for complex tasks in hazardous conditions.

Circuit

22,829 views • 1 year ago

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

Xiao Ma

46,323 views • 1 year ago

The Real-to-Sim part of R2S2R transforms physical robots, sensors, environments, objects, and interactions into simulations that preserve task-relevant observations and dynamics - not only how the world looks, but how it acts when the robot interacts with it. These results set a new bar for sim-real alignment in robot manipulation.

The Real-to-Sim part of R2S2R transforms physical robots, sensors, environments, objects, and interactions into simulations that preserve task-relevant observations and dynamics - not only how the world looks, but how it acts when the robot interacts with it. These results set a new bar for sim-real alignment in robot manipulation.

Fei-Fei Li

25,199 views • 25 days ago

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.

Lukas Ziegler

169,472 views • 1 year ago

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.

Sawyer Merritt

69,381 views • 1 year ago

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?

Lukas Ziegler

61,610 views • 1 year ago

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:

Ilir Aliu

51,490 views • 1 year ago

NEWS: DeepMind teaches robots to play soccer. In a paper released today Google's DeepMind details how they used Deep Reinforcement Learning (Deep RL) to train low-cost, miniature humanoid robots in dynamic environments, allowing them to play a simplified one-versus-one (1v1) soccer game. The robots, equipped with 20 actuated joints, were initially trained in simulation using the MuJoCo physics engine. Through this training, they learned robust and dynamic movement skills like rapid fall recovery, walking, turning, and kicking. The robots seamlessly transitioned between these skills, even surpassing expectations, and developed a basic strategic understanding of the game. During matches, the trained robots demonstrated agile skills such as turning, kicking moving balls, and dynamic defensive blocking. They quickly combined these skills, showcasing their adaptability and outperforming scripted baselines. The robots walked faster, got up quicker, and kicked faster than their counterparts. Individual skills were first trained in isolation within the simulation environment, and then composed in a self-play setting. The robots successfully transferred these skills to real-world scenarios. Full paper can be found here:

NEWS: DeepMind teaches robots to play soccer. In a paper released today Google's DeepMind details how they used Deep Reinforcement Learning (Deep RL) to train low-cost, miniature humanoid robots in dynamic environments, allowing them to play a simplified one-versus-one (1v1) soccer game. The robots, equipped with 20 actuated joints, were initially trained in simulation using the MuJoCo physics engine. Through this training, they learned robust and dynamic movement skills like rapid fall recovery, walking, turning, and kicking. The robots seamlessly transitioned between these skills, even surpassing expectations, and developed a basic strategic understanding of the game. During matches, the trained robots demonstrated agile skills such as turning, kicking moving balls, and dynamic defensive blocking. They quickly combined these skills, showcasing their adaptability and outperforming scripted baselines. The robots walked faster, got up quicker, and kicked faster than their counterparts. Individual skills were first trained in isolation within the simulation environment, and then composed in a self-play setting. The robots successfully transferred these skills to real-world scenarios. Full paper can be found here:

Dave Lee

372,039 views • 3 years ago

Running legged robots with AI. Today we met @swiss_mile, the team connecting AI with the physical world using autonomous machines. An ideal mobile robot needs to move quickly, overcoming obstacles while carrying heavy loads—something current robots still struggle with. Many robots haven't been widely adopted due to issues with real-world problem-solving. To address this, the Swiss-mile team have created a solution that combines robots with wheels and legs, powered by advanced AI. These robots can walk, drive, stand, open doors, and even handle packages. By integrating these features, the team are on a mission to push automation forward beyond stationary robotic arms and have raised $22m in Seed Funding from Jeff Bezos, HongShan, Amazon Industrial Innovation Fund and Armada Investment to do so.

Running legged robots with AI. Today we met @swiss_mile, the team connecting AI with the physical world using autonomous machines. An ideal mobile robot needs to move quickly, overcoming obstacles while carrying heavy loads—something current robots still struggle with. Many robots haven't been widely adopted due to issues with real-world problem-solving. To address this, the Swiss-mile team have created a solution that combines robots with wheels and legs, powered by advanced AI. These robots can walk, drive, stand, open doors, and even handle packages. By integrating these features, the team are on a mission to push automation forward beyond stationary robotic arms and have raised $22m in Seed Funding from Jeff Bezos, HongShan, Amazon Industrial Innovation Fund and Armada Investment to do so.

Circuit

17,876 views • 1 year ago

Robots on solar farms! ☀️ These Luminous robots assist in constructing solar farms, minimizing the need for manual handling of heavy parts. To reduce transport damage, module manufacturers are now using a "vertically stacked" standard, but this makes automation more complex. But these robots can still handle them! 🦾 Since large-format modules are heavy and long, robotic arms must handle the weight, balance the load, and move safely while avoiding collisions and external forces like wind.

Robots on solar farms! ☀️ These Luminous robots assist in constructing solar farms, minimizing the need for manual handling of heavy parts. To reduce transport damage, module manufacturers are now using a "vertically stacked" standard, but this makes automation more complex. But these robots can still handle them! 🦾 Since large-format modules are heavy and long, robotic arms must handle the weight, balance the load, and move safely while avoiding collisions and external forces like wind.

Lukas Ziegler

21,731 views • 1 year ago

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

20,898 views • 6 months ago

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐

📢 Announcing one of the most exciting works from us this year on scalable robot policy evaluation through real-to-sim transfer, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is orders of magnitude slower than the development of language agents. We propose a new framework where simulation performance strongly correlates with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing state-of-the-art correlation between simulation and reality for deformable object manipulation. It provides a scalable and reproducible evaluation engine for robot learning. 🌐

Yunzhu Li

39,960 views • 9 months ago

So, it’s happening. Chinese companies are starting to mass-produce advanced robots. 🇨🇳 DEEP Robotics recently gave us a glimpse into their quadruped robot production line, clear evidence that China intends to lead the global robotics race. Hundreds of units are already being tested, soon to leave factory floors and make their way to new owners in Europe and the US. It is good to be aware of the current state of the robotics market, especially since it has been growing rapidly for several years.

So, it’s happening. Chinese companies are starting to mass-produce advanced robots. 🇨🇳 DEEP Robotics recently gave us a glimpse into their quadruped robot production line, clear evidence that China intends to lead the global robotics race. Hundreds of units are already being tested, soon to leave factory floors and make their way to new owners in Europe and the US. It is good to be aware of the current state of the robotics market, especially since it has been growing rapidly for several years.

Lukas Ziegler

185,348 views • 11 months ago