Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

🔥 Hot release: Aloha unleashed World first demonstration of a robot able to tie shoelaces or hang t-shirts autonomously! They trained a diffusion policy at scale: 26,000 demonstrations over 5 tasks on Aloha 2 robot Retweet if you'd like them to open-source 😝 (video x4) 1/🧵

Remi Cadene

21,662 subscribers

198,426 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie Kunst Bildung

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Remi Cadene

Remi Cadenevor 1 Jahr

2/ Diffusion policy is trained with imitation learning only. No reinforcement learning. Neural architecture is inspired by ACT (but without Conditional VAE encoder). Base model has 217 millions learnable parameters

Profilbild von Remi Cadene

Remi Cadenevor 1 Jahr

3/ Standard training strategy: Batch size of 256 2 millions steps Adam with weight decay 0.001 linear learning rate warmup for 5000 steps followed by constant rate of 1e-4 Trained with JAX using 64 TPUv5e (how many hours of training? ^^)

Profilbild von Remi Cadene

Remi Cadenevor 1 Jahr

4/ Success rate is high overall! Is brute force imitation learning the way to go to reach 99%?

Profilbild von Remi Cadene

Remi Cadenevor 1 Jahr

5/ Important paper! I really hope they open source their datasets and models 🤓

Profilbild von sierra catalina

sierra catalinavor 1 Jahr

omg. I can't wait to see y'all at the beach. age of abundance is coming.

Profilbild von HackerTwins

HackerTwinsvor 1 Jahr

The best video would be two of these arms assemble a 3rd one, and then it powers on and starts assembling a 4th

Profilbild von Nils Ingelhag

Nils Ingelhagvor 1 Jahr

Impressive! But 1x speed? If your shoelaces are made of lead maybe ;)

Profilbild von AdrienBufort

AdrienBufortvor 1 Jahr

It feels like the next thing in AI is robotic dexterity with human prompt as direct command :)

Profilbild von Nex - AI Summarizer (100% FREE)

Nex - AI Summarizer (100% FREE)vor 1 Jahr

Competent robots!

Profilbild von Scott Tindle

Scott Tindlevor 1 Jahr

If you still don't believe the robot revolution is coming you're not paying attention.

Ähnliche Videos

We release Cosmos Policy 💫: a state-of-the-art robot policy built on a video diffusion model backbone. - policy + world model + value function — in 1 model - no architectural changes to the base video model - SOTA in LIBERO (98.5%), RoboCasa (67.1%), & ALOHA tasks (93.6%) 🧵👇

We release Cosmos Policy 💫: a state-of-the-art robot policy built on a video diffusion model backbone. - policy + world model + value function — in 1 model - no architectural changes to the base video model - SOTA in LIBERO (98.5%), RoboCasa (67.1%), & ALOHA tasks (93.6%) 🧵👇

Moo Jin Kim

149,398 Aufrufe • vor 6 Monaten

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Jianlan Luo

63,831 Aufrufe • vor 2 Monaten

Sharing a few more demos of our first #LargeBehaviorModel (LBM) at TRI. 1/🍎This model enables a robot to learned to core and cut an apple into multiple slices autonomously. We trained our diffusion-based LBM on almost 1,700 hours of robot data, conducted 1,800 real-world evaluation rollouts, and ran over 47,000 simulation rollouts to rigorously study its capabilities. Learn more here:

Sharing a few more demos of our first #LargeBehaviorModel (LBM) at TRI. 1/🍎This model enables a robot to learned to core and cut an apple into multiple slices autonomously. We trained our diffusion-based LBM on almost 1,700 hours of robot data, conducted 1,800 real-world evaluation rollouts, and ran over 47,000 simulation rollouts to rigorously study its capabilities. Learn more here:

Zubair Irshad

12,202 Aufrufe • vor 1 Jahr

This is a single uncut video, showing a robot learning several tasks instantly, after just one demonstration each ... This is possible because we've now been able to achieve in-context learning for everyday robotics tasks, and I'm very excited to announce our latest paper: 🎆 Instant Policy: In-Context Imitation Learning via Graph Diffusion 🎆 (1/6) 🧵👇

This is a single uncut video, showing a robot learning several tasks instantly, after just one demonstration each ... This is possible because we've now been able to achieve in-context learning for everyday robotics tasks, and I'm very excited to announce our latest paper: 🎆 Instant Policy: In-Context Imitation Learning via Graph Diffusion 🎆 (1/6) 🧵👇

Edward Johns

74,683 Aufrufe • vor 1 Jahr

Policies trained on real robot data via imitation can be surprisingly capable. But for domains like dexterous manipulation, they are often not quite good enough: they move slowly, miss grasps, make unreliable contact, and fail under small perturbations. Can we improve them without any additional data collection on the real robot? In SCORE, we show that we can improve real-world diffusion/flow policies cheaply by using simulation to simply learn how to steer them on deployment. This leads to large gains in real-world success and speed across a variety of tasks, without requiring additional real-world experience: 🧵 (1/10)

Policies trained on real robot data via imitation can be surprisingly capable. But for domains like dexterous manipulation, they are often not quite good enough: they move slowly, miss grasps, make unreliable contact, and fail under small perturbations. Can we improve them without any additional data collection on the real robot? In SCORE, we show that we can improve real-world diffusion/flow policies cheaply by using simulation to simply learn how to steer them on deployment. This leads to large gains in real-world success and speed across a variety of tasks, without requiring additional real-world experience: 🧵 (1/10)

Abhishek Gupta

34,406 Aufrufe • vor 1 Monat

🚀 We’re excited to announce LingBot-VA, a new state-of-the-art robot policy model from Robbyant ! LingBot-VA is built on a causal, autoregressive video-action world model for generalist robot control. Highlights: (1) First unified autoregressive video-action world model for robot control (2) Low-latency inference with a new asynchronous execution pipeline (3) SOTA on RoboTwin (92.9%, firstever > 90%) and LIBERO (98.5%) (4) +20% over π0.5 on challenging real-world long-horizon & high-precision tasks

🚀 We’re excited to announce LingBot-VA, a new state-of-the-art robot policy model from Robbyant ! LingBot-VA is built on a causal, autoregressive video-action world model for generalist robot control. Highlights: (1) First unified autoregressive video-action world model for robot control (2) Low-latency inference with a new asynchronous execution pipeline (3) SOTA on RoboTwin (92.9%, firstever > 90%) and LIBERO (98.5%) (4) +20% over π0.5 on challenging real-world long-horizon & high-precision tasks

Yinghao Xu

46,962 Aufrufe • vor 6 Monaten

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad

These guys trained a robot with water bottles, and the robot learned to pick up other objects it had never practiced on. They did this using reinforcement learning: 1. The robot had to pick a bottle out of a messy tote 2. Then it had to hand the bottle to a person 3. Only water bottles. Nothing else. After they finished training, they tested the robot on different objects: • On bottles, the success went from 80% to 98% • On snack cans, the robot's throughput rose 40% • On soft bags, the robot's performance improved by 13% even though they're deformable and much harder to handle than rigid objects. Snack cans and soft bags have different shapes and weights, and the robot was never trained on them. This was just one of three use cases they tested. Every hour we spend teaching a robot costs a ton of money. If they only get better at handling the one object they practice on, reinforcement learning will never be worth doing. But if the gains carry over, we could focus on teaching the hardest possible tasks and let the robot apply that knowledge elsewhere. Here is the full write-up by Humanoid on the infrastructure they used and what they learned: #ad

Santiago

13,952 Aufrufe • vor 21 Tagen

In my first week at Generalist, I trained a robot to pour liquids using GEN-1 🤖💧 I wanted to challenge the robot with a non-rigid manipulation task, so liquid felt like the perfect choice. The task involved: - unscrewing the bottle cap - pouring liquid into espresso glasses - rebalancing uneven pours Best of all, the robot was able to complete the task fully autonomously 3 times in a row (out of 3)! Pour-fect 😉 Excited for the journey ahead and grateful to be building alongside such an incredible team!

In my first week at Generalist, I trained a robot to pour liquids using GEN-1 🤖💧 I wanted to challenge the robot with a non-rigid manipulation task, so liquid felt like the perfect choice. The task involved: - unscrewing the bottle cap - pouring liquid into espresso glasses - rebalancing uneven pours Best of all, the robot was able to complete the task fully autonomously 3 times in a row (out of 3)! Pour-fect 😉 Excited for the journey ahead and grateful to be building alongside such an incredible team!

Kylie Ying

37,327 Aufrufe • vor 2 Monaten

NAVIAI-I3, a more robust humanoid robot, was launched by the Zhejiang Humanoid Robotics Innovation Center. Standing 1.8 meters tall, it can run like a human and walk on various terrains. This humanoid robot appears to be primarily designed for specialized industrial tasks, such as security patrols in underground parking lots. They previously launched the lightweight bipedal NVAI-2 and a wheeled humanoid robot. The bipedal robot is used in manufacturing operations, while the wheeled humanoid robot is used for unstructured, flexible tasks such as warehouse sorting, supermarket shelving, and office cleaning. We filmed this demonstration at the World Robot Conference; you can see it in the comments section.

NAVIAI-I3, a more robust humanoid robot, was launched by the Zhejiang Humanoid Robotics Innovation Center. Standing 1.8 meters tall, it can run like a human and walk on various terrains. This humanoid robot appears to be primarily designed for specialized industrial tasks, such as security patrols in underground parking lots. They previously launched the lightweight bipedal NVAI-2 and a wheeled humanoid robot. The bipedal robot is used in manufacturing operations, while the wheeled humanoid robot is used for unstructured, flexible tasks such as warehouse sorting, supermarket shelving, and office cleaning. We filmed this demonstration at the World Robot Conference; you can see it in the comments section.

CyberRobo

15,521 Aufrufe • vor 8 Monaten

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,864 Aufrufe • vor 6 Monaten

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏

Ilir Aliu

46,285 Aufrufe • vor 1 Jahr

The first open-source unified world model for scalable robot manipulation: 5B-parameter open-source unified video-action world model that combines policy and world modeling to generate robot actions, predict future visuals, and evaluate task progress from observations, language, and state. The model is trained on 27.3K hours of heterogeneous data (17.8K real-robot teleop, 6.5K UMI demos, 3K egocentric human videos), enabling it to perform complex manipulation tasks like faucet connecting, bag packing, and toolbox storing as shown in demo videos. The approach supports test-time action refinement and points toward deployment-driven continuous improvement via fleet data. Thanks for sharing, Jianlan Luo (Jianlan Luo)! 📌 Resource links for τ0-WM: • Project page: • GitHub (code): • Hugging Face (model weights): • Paper (PDF): ——- Weekly robotics and AI insights. Subscribe free:

The first open-source unified world model for scalable robot manipulation: 5B-parameter open-source unified video-action world model that combines policy and world modeling to generate robot actions, predict future visuals, and evaluate task progress from observations, language, and state. The model is trained on 27.3K hours of heterogeneous data (17.8K real-robot teleop, 6.5K UMI demos, 3K egocentric human videos), enabling it to perform complex manipulation tasks like faucet connecting, bag packing, and toolbox storing as shown in demo videos. The approach supports test-time action refinement and points toward deployment-driven continuous improvement via fleet data. Thanks for sharing, Jianlan Luo (Jianlan Luo)! 📌 Resource links for τ0-WM: • Project page: • GitHub (code): • Hugging Face (model weights): • Paper (PDF): ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

28,276 Aufrufe • vor 2 Monaten

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure

🚨🇺🇸FIGURE ROBOT LEARNS TO WALK IN HOURS Turns out you can teach a robot to walk—if you’ve got a few years to burn in a physics simulator (or a few hours of GPU time). Figure’s team trained a neural network using reinforcement learning to mimic natural human locomotion, and the wild part is it worked on the first try with real hardware. No tuning. No fiddling. Just plug and strut. Thanks to sim-to-real transfer and some sneaky domain randomization, the robot didn’t just wobble out of the lab—it walked like it had somewhere to be. Next up: running. Not terrifying at all. Source: Figure

Mario Nawfal

117,866 Aufrufe • vor 1 Jahr

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 Aufrufe • vor 4 Monaten

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

Zhi (Leo) Wang

50,541 Aufrufe • vor 1 Monat

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Abhishek Gupta

11,994 Aufrufe • vor 1 Jahr

Enough with LLMs - exciting things are happening in the world of atoms. This is Stanford ALOHA, a low-cost and agile robot platform. The whole system is open-source (!!): hardware design, CAD models for 3D printing, simulator, and training code. Time to graft a physical arm onto GPTs 🦾 Led by my friend Tony Zhao at Stanford, and advised by Chelsea Finn Sergey Levine Vikash Kumar. Project page: If you don't want to 3D print your own components, you can buy the setup at

Enough with LLMs - exciting things are happening in the world of atoms. This is Stanford ALOHA, a low-cost and agile robot platform. The whole system is open-source (!!): hardware design, CAD models for 3D printing, simulator, and training code. Time to graft a physical arm onto GPTs 🦾 Led by my friend Tony Zhao at Stanford, and advised by Chelsea Finn Sergey Levine Vikash Kumar. Project page: If you don't want to 3D print your own components, you can buy the setup at

Jim Fan

818,539 Aufrufe • vor 3 Jahren

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:

Ilir Aliu

18,210 Aufrufe • vor 6 Monaten

Israel-based Mentee Robotics has demonstrated a logistics workflow: two MenteeBot V3 humanoids work autonomously to pick and place totes. A Modular Agent System is preferred because it favors real-world robustness and lower compute needs over the End-to-End VLA model. Its architecture is composed of three components: - LLM Planner: Converts instructions into executable Robotic API Language code for reliable task decomposition and error handling. - Perception Stack: Uses pre-trained models (NeRF/3DGS, distilled vision) for scene understanding and navigation. - Control Policies: Reinforcement Learning (RL) models, trained at scale via Sim2Real, generate motor commands, enabling high-accuracy mobile manipulation. Crucially, the robot learns new tasks from a single demonstration in hours. Object tracking uses 3D geometry (STL/URDF) tracked in the video to define the RL reward function. Training is optimized using 'Automatic Curriculum Learning', which autonomously adjusts task difficulty based on robot performance, eliminating manual engineering. All computation runs onboard.

Israel-based Mentee Robotics has demonstrated a logistics workflow: two MenteeBot V3 humanoids work autonomously to pick and place totes. A Modular Agent System is preferred because it favors real-world robustness and lower compute needs over the End-to-End VLA model. Its architecture is composed of three components: - LLM Planner: Converts instructions into executable Robotic API Language code for reliable task decomposition and error handling. - Perception Stack: Uses pre-trained models (NeRF/3DGS, distilled vision) for scene understanding and navigation. - Control Policies: Reinforcement Learning (RL) models, trained at scale via Sim2Real, generate motor commands, enabling high-accuracy mobile manipulation. Crucially, the robot learns new tasks from a single demonstration in hours. Object tracking uses 3D geometry (STL/URDF) tracked in the video to define the RL reward function. Training is optimized using 'Automatic Curriculum Learning', which autonomously adjusts task difficulty based on robot performance, eliminating manual engineering. All computation runs onboard.

The Humanoid Hub

15,729 Aufrufe • vor 8 Monaten