Loading video...

Video Failed to Load

Go Home

Long horizon robotics is hard. Robots drift, forget the goal, and fall apart when a task takes hundreds of steps. This small independent team hit rank 1 on Stanford’s Behavior1K benchmark by Fei Fei Li’s lab… Ahead of teams from NVIDIA, CMU, and several big labs. What they found...

IlirAliu_'s profile picture

Ilir Aliu

53,813 subscribers

12,902 views • 9 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:
0:47

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,881 views • 10 months ago

I don't know if you really understand what these people just did... They built an AI that looks at a few casual real-world videos or photos and instantly turns them into a full 3D playground. You can MOVE THE CAMERA HOWEVER YOU WANT, change the lighting, move objects around, and even let physics play out like in the real world. View events from impossible angles because Atlas knows what was there. This is massive! Let's take robotics. Until now training robots meant collecting tons of expensive real-world data. Now a handful of recordings can become thousands of different simulated situations. Robots can practice the same task in new rooms, with different objects and lighting without anyone having to film it all again. For the industry it means we can train robots way faster and cheaper. Home robots, warehouse robots, whatever... they can learn new environments at scale instead of staying stuck in the lab. This is one of the first real bridges between “cool AI video models” and actual physical machines that have to work in the real world. Blog: The team behind World Labs: Fei-Fei Li, Justin Johnson, Ben Mildenhall and Yunzhu Li. The video is from David Pantera. Follow for more insights into robotics and AI. —— Weekly robotics and AI insights. Subscribe free:
0:49

I don't know if you really understand what these people just did... They built an AI that looks at a few casual real-world videos or photos and instantly turns them into a full 3D playground. You can MOVE THE CAMERA HOWEVER YOU WANT, change the lighting, move objects around, and even let physics play out like in the real world. View events from impossible angles because Atlas knows what was there. This is massive! Let's take robotics. Until now training robots meant collecting tons of expensive real-world data. Now a handful of recordings can become thousands of different simulated situations. Robots can practice the same task in new rooms, with different objects and lighting without anyone having to film it all again. For the industry it means we can train robots way faster and cheaper. Home robots, warehouse robots, whatever... they can learn new environments at scale instead of staying stuck in the lab. This is one of the first real bridges between “cool AI video models” and actual physical machines that have to work in the real world. Blog: The team behind World Labs: Fei-Fei Li, Justin Johnson, Ben Mildenhall and Yunzhu Li. The video is from David Pantera. Follow for more insights into robotics and AI. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

272,539 views • 18 days ago

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏
0:50

The robot is learning several novel tasks instantly, after just ONE demonstration each... Instant Policy makes it possible: no extra training, no weight updates, just pure in-context learning. It just got accepted at ICLR 2025, and it’s changing how robots learn. With just a single demo, a robot can pick up a new task and start performing it right away. Why this is a big deal: ✅ Learns tasks instantly with just one or a few demonstrations ✅ Improves over time as more demonstrations are given ✅ Uses simulation-based training with “pseudo-demonstrations” for scalability ✅ Can transfer skills across different robots and even follow language-defined tasks It brings in-context learning to robotics, opening up new possibilities for flexible, real-world automation. You can try it yourself: code and weights are available at • • • • Thank you to Edward Johns, Director of the Robot Learning Lab at Imperial College for sharing their work! 🙏

Ilir Aliu

46,285 views • 1 year ago

Most robotics teams can make a robot move… very few can make it work every day in the real world! @roboforce_inc was just featured in Jensen Huang’s GTC keynote, not for a demo but for a robot they claim is already working in the field. Their system, TITAN, is built for industrial environments: ✅ 40 kg dual-arm payload ✅ millimeter-level precision ✅ 8 hours runtime ✅ early pilots in energy, logistics, and manufacturing They call it Physical AI: The focus is on getting robots to do the kind of labor that still depends on people: repetitive, risky, and physical. IMO, what stands out isn’t the specs. It’s the claim of 11,000 robot orders❗️ If even a fraction of that turns into real deployments, it would make RoboForce one of the most widely scaled robot platforms in the world!!?? The open question is whether this shift from prototype to production is real or just the next phase of robotics storytelling. It looks very promising to me. It’s also interesting how their founder openly pushes back against the humanoid trend, arguing for wheels over legs in real industrial settings. It’s an unusual perspective in a time when humanoids get most of the attention. If they are right, the next humanoid robotics wave might look less like a human and more like an industrial version. Seen at —- Weekly robotics and AI insights. Subscribe free:
0:29

Most robotics teams can make a robot move… very few can make it work every day in the real world! @roboforce_inc was just featured in Jensen Huang’s GTC keynote, not for a demo but for a robot they claim is already working in the field. Their system, TITAN, is built for industrial environments: ✅ 40 kg dual-arm payload ✅ millimeter-level precision ✅ 8 hours runtime ✅ early pilots in energy, logistics, and manufacturing They call it Physical AI: The focus is on getting robots to do the kind of labor that still depends on people: repetitive, risky, and physical. IMO, what stands out isn’t the specs. It’s the claim of 11,000 robot orders❗️ If even a fraction of that turns into real deployments, it would make RoboForce one of the most widely scaled robot platforms in the world!!?? The open question is whether this shift from prototype to production is real or just the next phase of robotics storytelling. It looks very promising to me. It’s also interesting how their founder openly pushes back against the humanoid trend, arguing for wheels over legs in real industrial settings. It’s an unusual perspective in a time when humanoids get most of the attention. If they are right, the next humanoid robotics wave might look less like a human and more like an industrial version. Seen at —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

27,741 views • 10 months ago

A simple idea. Let robots collect the data that current foundation models are missing. A robot that gets better by doing real work in the real world. For two weeks in the Stanford East Asia Library, Scanford scanned shelves, helped librarians, and improved the vision language model it depends on. The idea is very simple: Robots do useful work. They gather the real world data foundation models never see online. They fine tune their own model They go back out stronger A full loop. What they found in deployment: ✅ 2103 shelves scanned with multilingual, faded, occluded book spines ✅ 18.7 hours of librarian time saved ✅ Book ID accuracy jumped from 32.0 percent to 71.8 percent ✅ English OCR improved from 24.8 percent to 46.6 percent ✅ Chinese OCR improved from 30.8 percent to 38.0 percent The most interesting part is the shift. Robots do not only consume foundation models. They create the data these models are missing. A clean robot powered data flywheel. Work. Collect. Fine tune. Repeat. Thanks for sharing, Jenn Grannen! If you want the full write up: 📍Website: Paper: —- Weekly robotics and AI insights. Subscribe free:
0:44

A simple idea. Let robots collect the data that current foundation models are missing. A robot that gets better by doing real work in the real world. For two weeks in the Stanford East Asia Library, Scanford scanned shelves, helped librarians, and improved the vision language model it depends on. The idea is very simple: Robots do useful work. They gather the real world data foundation models never see online. They fine tune their own model They go back out stronger A full loop. What they found in deployment: ✅ 2103 shelves scanned with multilingual, faded, occluded book spines ✅ 18.7 hours of librarian time saved ✅ Book ID accuracy jumped from 32.0 percent to 71.8 percent ✅ English OCR improved from 24.8 percent to 46.6 percent ✅ Chinese OCR improved from 30.8 percent to 38.0 percent The most interesting part is the shift. Robots do not only consume foundation models. They create the data these models are missing. A clean robot powered data flywheel. Work. Collect. Fine tune. Repeat. Thanks for sharing, Jenn Grannen! If you want the full write up: 📍Website: Paper: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

44,660 views • 9 months ago

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:
0:29

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,127 views • 7 months ago

The robot flipped a pancake nobody taught it! 🥞 Skild AI team assumed pancake flipping had to be somewhere in the training data. So they searched. Millions of hours of pre-training data. Nothing. S1 inferred the whole task from a single human demonstration. That's their new general robot model, built as an in-context learner from the ground up. Every new robot task today starts with days of teleoperation and a fine-tuning run on a specialist policy. S1 skips all of it. Much like a language model, it never updates its weights to learn a new task. The demonstration enters the context window, and the policy uses it to decide what to do next. → Ten-minute tasks it was never trained on, composed from primitives learned in pre-training: a new style of coffee, potting a plant, frying pancakes. → Soil and pots arrived at their office at 8:54 PM. The robot was running the task autonomously by 9:27 PM. → Slide objects away mid-reach, swap them, change the lighting, it still finishes. → The prompt waters a plant with a watering can, but only a cup is available. It uses the cup. It doesn't rigidly replay what it saw, but it recovers from its own errors, and sometimes executes with more precision than the demonstrator, when the human fumbles an egg and makes a mess, S1 performs the same step cleanly. The demonstration is a specification of the goal, and not a trajectory to copy. On unseen tasks after 100K hours of pre-training: language-prompted VLAs reach 9%. Their new model reaches 66%. It's already deploying with industrial partners, with a wider rollout over the coming months. Congrats Deepak Pathak and team behind this! 😮‍💨 🔗 Link to their latest blog: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
1:34

The robot flipped a pancake nobody taught it! 🥞 Skild AI team assumed pancake flipping had to be somewhere in the training data. So they searched. Millions of hours of pre-training data. Nothing. S1 inferred the whole task from a single human demonstration. That's their new general robot model, built as an in-context learner from the ground up. Every new robot task today starts with days of teleoperation and a fine-tuning run on a specialist policy. S1 skips all of it. Much like a language model, it never updates its weights to learn a new task. The demonstration enters the context window, and the policy uses it to decide what to do next. → Ten-minute tasks it was never trained on, composed from primitives learned in pre-training: a new style of coffee, potting a plant, frying pancakes. → Soil and pots arrived at their office at 8:54 PM. The robot was running the task autonomously by 9:27 PM. → Slide objects away mid-reach, swap them, change the lighting, it still finishes. → The prompt waters a plant with a watering can, but only a cup is available. It uses the cup. It doesn't rigidly replay what it saw, but it recovers from its own errors, and sometimes executes with more precision than the demonstrator, when the human fumbles an egg and makes a mess, S1 performs the same step cleanly. The demonstration is a specification of the goal, and not a trajectory to copy. On unseen tasks after 100K hours of pre-training: language-prompted VLAs reach 9%. Their new model reaches 66%. It's already deploying with industrial partners, with a wider rollout over the coming months. Congrats Deepak Pathak and team behind this! 😮‍💨 🔗 Link to their latest blog: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

11,406 views • 25 days ago

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:
1:52

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:

Ilir Aliu

51,526 views • 1 year ago

A team tested Pi0, Pi0 Fast, Gr00t, and ACT on real robot arms in manufacturing tasks. (🔖 Bookmark this for later!) The task was precise: place thin rectangular frames from a messy stack into a holder. The team fine-tuned each model on 100 real trajectories and compared training time, inference speed, motion quality, and success rates. ⬇️ Here’s a breakdown of what they found Pi0 (Original) ✅ Strongest overall performance in precise pick-and-place ✅ High success rate even in edge cases ✅ Longest training time (~11 hours, ~$30 per run) ✅ Inference time of 80 ms causes short pauses between actions Despite delays, it handles complex scenarios well… solid for high-precision tasks, but slow to train. Gr00t ✅ Trains fast (~2 hours, ~$5 per run) ✅ Performs almost as well as Pi0 on large-object tasks ✅ Struggles with fine precision; random movement in some trials ✅ More training didn’t fix jitter or random offsets Best suited for tasks where exact precision isn’t critical. Not ready for manufacturing-grade accuracy without more tuning. Pi0 Fast ✅ Promised faster training, but results were underwhelming ✅ Training at 6 hours still showed low success rates ✅ Inference was slower than expected ✅ Not reliable for generalizing even slightly new tasks Currently too unstable for real-world deployment. Doesn’t live up to the “Fast” name yet. ACT (Baseline) ✅ 200MB model—lightweight, but limited ✅ Struggles with stacked objects or ambiguous scenes ✅ Success rates around 70% in best-case setups ✅ Can’t match newer models on precision or generalization Still a solid baseline, but clearly a generation behind in robustness. 🚨 Extra Notes All newer models share a common issue: •Inference takes longer than a frame (80 ms vs 33 ms), so robots “pause” between chunks. •This results in jittery movements, but not a dealbreaker unless tasks are time-sensitive. Language-conditioned tasks also fell short: after training on two labeled tasks, the model couldn’t generalize to a third unseen combination using only text prompts. ✅ The good news? These models adapt well to new robot arms with quick fine-tuning. ❌ The bad news? There’s still no plug-and-play solution for improving performance after deployment. Reinforcement learning or DAgger-style data collection during real-world operation may be the next big step, something many teams in robotics are actively working on.
0:51

A team tested Pi0, Pi0 Fast, Gr00t, and ACT on real robot arms in manufacturing tasks. (🔖 Bookmark this for later!) The task was precise: place thin rectangular frames from a messy stack into a holder. The team fine-tuned each model on 100 real trajectories and compared training time, inference speed, motion quality, and success rates. ⬇️ Here’s a breakdown of what they found Pi0 (Original) ✅ Strongest overall performance in precise pick-and-place ✅ High success rate even in edge cases ✅ Longest training time (~11 hours, ~$30 per run) ✅ Inference time of 80 ms causes short pauses between actions Despite delays, it handles complex scenarios well… solid for high-precision tasks, but slow to train. Gr00t ✅ Trains fast (~2 hours, ~$5 per run) ✅ Performs almost as well as Pi0 on large-object tasks ✅ Struggles with fine precision; random movement in some trials ✅ More training didn’t fix jitter or random offsets Best suited for tasks where exact precision isn’t critical. Not ready for manufacturing-grade accuracy without more tuning. Pi0 Fast ✅ Promised faster training, but results were underwhelming ✅ Training at 6 hours still showed low success rates ✅ Inference was slower than expected ✅ Not reliable for generalizing even slightly new tasks Currently too unstable for real-world deployment. Doesn’t live up to the “Fast” name yet. ACT (Baseline) ✅ 200MB model—lightweight, but limited ✅ Struggles with stacked objects or ambiguous scenes ✅ Success rates around 70% in best-case setups ✅ Can’t match newer models on precision or generalization Still a solid baseline, but clearly a generation behind in robustness. 🚨 Extra Notes All newer models share a common issue: •Inference takes longer than a frame (80 ms vs 33 ms), so robots “pause” between chunks. •This results in jittery movements, but not a dealbreaker unless tasks are time-sensitive. Language-conditioned tasks also fell short: after training on two labeled tasks, the model couldn’t generalize to a third unseen combination using only text prompts. ✅ The good news? These models adapt well to new robot arms with quick fine-tuning. ❌ The bad news? There’s still no plug-and-play solution for improving performance after deployment. Reinforcement learning or DAgger-style data collection during real-world operation may be the next big step, something many teams in robotics are actively working on.

Ilir Aliu

21,844 views • 1 year ago

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:
0:41

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

14,222 views • 4 months ago

A policy that teaches robot hands to touch things the way humans do... not just grab and move, but feel and adjust in real time. Robot manipulation research often stops at picking up objects and placing them. CGP goes further: it handles tasks like opening jars, flipping objects in-hand, wiping dishes, and grasping fragile eggs, the kind of dexterous, contact-rich skills that require constant micro-adjustments based on what the fingers are actually feeling. The robot doesn't just see what it's doing; it predicts what contact should feel like at each step, then checks whether reality matches the prediction. If a finger is slipping, the policy knows before the object drops. Works on real robot hands (both 4-finger and 5-finger designs) with tactile sensors embedded in the fingertips Robust to visual distractions! The robot keeps flipping a box correctly even when the camera view is disrupted, because it's grounding decisions in touch, not just vision. Baseline policies without contact grounding fail in predictable ways: slipping mid-task, incomplete motions, loss of grasp, CGP avoids these This is a meaningful step toward robots that can handle the physical world with the kind of reliable, adaptive grip that humans take for granted. Relevant for manufacturing, logistics, assistive robotics, and anywhere fragile or irregular objects need to be handled carefully. Published at RSS 2026, developed with Meta Reality Labs Research. Thanks for sharing, Zhengtong Xu / Zhengtong Xu ——- Weekly robotics and AI insights. Subscribe free:
0:36

A policy that teaches robot hands to touch things the way humans do... not just grab and move, but feel and adjust in real time. Robot manipulation research often stops at picking up objects and placing them. CGP goes further: it handles tasks like opening jars, flipping objects in-hand, wiping dishes, and grasping fragile eggs, the kind of dexterous, contact-rich skills that require constant micro-adjustments based on what the fingers are actually feeling. The robot doesn't just see what it's doing; it predicts what contact should feel like at each step, then checks whether reality matches the prediction. If a finger is slipping, the policy knows before the object drops. Works on real robot hands (both 4-finger and 5-finger designs) with tactile sensors embedded in the fingertips Robust to visual distractions! The robot keeps flipping a box correctly even when the camera view is disrupted, because it's grounding decisions in touch, not just vision. Baseline policies without contact grounding fail in predictable ways: slipping mid-task, incomplete motions, loss of grasp, CGP avoids these This is a meaningful step toward robots that can handle the physical world with the kind of reliable, adaptive grip that humans take for granted. Relevant for manufacturing, logistics, assistive robotics, and anywhere fragile or irregular objects need to be handled carefully. Published at RSS 2026, developed with Meta Reality Labs Research. Thanks for sharing, Zhengtong Xu / Zhengtong Xu ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

12,854 views • 3 months ago

Is it really worth it joining a robotics hackathon? One year ago, thousands of people around the world spent a single weekend building AI robots using the open-source LeRobot stack. Many had never built a robot before. We sat down with the winners to ask what they made and where they are now. For many of them, that weekend changed everything, turning the intimidating field of AI robotics into an accessible hobby or even a full-time career. Here are a few key takeaways from their year-long journey: - You don't need a PhD to start. For many participants, this was their very first robot - One weekend can launch a career. Winning secured internships and full-time jobs - Community lowers the barrier to entry - Learning by doing is exponentially faster. Robotics still requires getting your hands dirty to truly progress Watch the full video to see their projects and hear their advice for taking the first step. clem 🤗, Thomas Wolf
7:26

Is it really worth it joining a robotics hackathon? One year ago, thousands of people around the world spent a single weekend building AI robots using the open-source LeRobot stack. Many had never built a robot before. We sat down with the winners to ask what they made and where they are now. For many of them, that weekend changed everything, turning the intimidating field of AI robotics into an accessible hobby or even a full-time career. Here are a few key takeaways from their year-long journey: - You don't need a PhD to start. For many participants, this was their very first robot - One weekend can launch a career. Winning secured internships and full-time jobs - Community lowers the barrier to entry - Learning by doing is exponentially faster. Robotics still requires getting your hands dirty to truly progress Watch the full video to see their projects and hear their advice for taking the first step. clem 🤗, Thomas Wolf

LeRobot

40,300 views • 2 months ago

NEW AXIS TASK GUIDE 👇 Task: Franka, Put the Charger into the Upper Drawer of the Cabinet. Home – Variant A The goal is simple: Open both cabinet drawers, pick up the charger and place it inside the upper drawer. First, pull both drawers completely open so the robot has enough access to the cabinet. Then identify and grab the charger carefully. After that, move the charger toward the upper drawer, not the lower one. Finally, place the charger inside the drawer and complete the task. 🔌 What makes this interesting is that the robot isn’t just moving an object from A to B. It has to understand the environment, drawer positions, object location and the final target before completing the action. This is actually similar to something we do every day opening a drawer, grabbing a charger and putting it away without even thinking about the steps. For a robot, these small actions require learning how different objects interact and how movements need to change depending on the environment. I also like that this task involves multiple actions in sequence, rather than one simple movement. That kind of sequential interaction can be important for robots operating in homes, offices and other real world environments. For me, this is where Axis Robotics becomes interesting, turning everyday human actions into structured robotic training data. Simple household task → complex robotic learning.
0:26

NEW AXIS TASK GUIDE 👇 Task: Franka, Put the Charger into the Upper Drawer of the Cabinet. Home – Variant A The goal is simple: Open both cabinet drawers, pick up the charger and place it inside the upper drawer. First, pull both drawers completely open so the robot has enough access to the cabinet. Then identify and grab the charger carefully. After that, move the charger toward the upper drawer, not the lower one. Finally, place the charger inside the drawer and complete the task. 🔌 What makes this interesting is that the robot isn’t just moving an object from A to B. It has to understand the environment, drawer positions, object location and the final target before completing the action. This is actually similar to something we do every day opening a drawer, grabbing a charger and putting it away without even thinking about the steps. For a robot, these small actions require learning how different objects interact and how movements need to change depending on the environment. I also like that this task involves multiple actions in sequence, rather than one simple movement. That kind of sequential interaction can be important for robots operating in homes, offices and other real world environments. For me, this is where Axis Robotics becomes interesting, turning everyday human actions into structured robotic training data. Simple household task → complex robotic learning.

SufianXFN

57,551 views • 8 days ago

Teleoperation today relies on WebRTC. That works for Zoom. Not for robotics. Is that statement provocative? WebRTC works for video calls. But not really for robots. I’m teleoperating from SF → London using Adamo’s stack. Zero buffering. No lag smoothing… just real-time control. Because WebRTC apps add buffers to handle: • packet loss • network jitter That’s fine when you’re talking. Not so much when you’re controlling a robot. Even small delays kill precision. So instead of adapting WebRTC… Adamo built a custom protocol + dedicated global infrastructure. Without buffering or artificial delay. A stable, real-time connection across the Atlantic. This is what people underestimate about robotics: Models are just one piece of the puzzle. But looking at the entire stack, there is so much more. And Sometimes you can’t reuse what exists. You have to build it from the ground up. Thank you, Sam Considine for letting me try, it nice seeing you during GTC last week! 👋 ——- Weekly robotics and AI insights. Subscribe free:
0:54

Teleoperation today relies on WebRTC. That works for Zoom. Not for robotics. Is that statement provocative? WebRTC works for video calls. But not really for robots. I’m teleoperating from SF → London using Adamo’s stack. Zero buffering. No lag smoothing… just real-time control. Because WebRTC apps add buffers to handle: • packet loss • network jitter That’s fine when you’re talking. Not so much when you’re controlling a robot. Even small delays kill precision. So instead of adapting WebRTC… Adamo built a custom protocol + dedicated global infrastructure. Without buffering or artificial delay. A stable, real-time connection across the Atlantic. This is what people underestimate about robotics: Models are just one piece of the puzzle. But looking at the entire stack, there is so much more. And Sometimes you can’t reuse what exists. You have to build it from the ground up. Thank you, Sam Considine for letting me try, it nice seeing you during GTC last week! 👋 ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

21,551 views • 5 months ago

My conversation with Sergey Levine (Sergey Levine). Sergey is the co-founder of Physical Intelligence -- a company building foundation models that can control any robot to do any task in any environment. The company's thesis is that generality is more scalable than specialization, meaning that a model trained across many different robots and tasks will ultimately outperform any system built to do one thing well (eg, just wash dishes). Sergey is a researcher by background, but I think you will appreciate how practical and commercially grounded this conversation is. We discuss: - Why changing a diaper will be the last task a robot masters - The simulation v. real-world data debate - How multimodal LLMs give robots common sense - Moravec's Paradox + Robot Olympics - Why robots can do long-horizon tasks now - A realistic timeline for robots in our homes I should note that I am an investor in Physical Intelligence -- I made the investment because I believe it is one of the most important companies tackling the problem of robotics. Enjoy! Timestamps: 0:00 Intro 2:39 Defining Physical Intelligence 5:19 The Challenge of Building General Models 6:34 The Stakes and Future of General Purpose Robotics 8:15 Pros and Cons of Humanoid Robots 10:12 Historical Milestones in Robotics Research 15:31 Combining Generative AI and Deep RL 21:24 Moravec's Paradox 25:33 Kitchen Robots 29:30 Simulation vs. Real-World Data 30:48 The Robot Olympics 36:31 The Physiological Reality of Embodiment 38:56 Controversies in the Robotics Community 44:18 What Makes a Great Researcher 48:27 How Businesses Should Prepare for Robotics 54:09 Tracking Progress Through Research Papers 57:02 The Next Step: Mid-Level Reasoning 1:02:00 The Kindest Thing
1:15:15

My conversation with Sergey Levine (Sergey Levine). Sergey is the co-founder of Physical Intelligence -- a company building foundation models that can control any robot to do any task in any environment. The company's thesis is that generality is more scalable than specialization, meaning that a model trained across many different robots and tasks will ultimately outperform any system built to do one thing well (eg, just wash dishes). Sergey is a researcher by background, but I think you will appreciate how practical and commercially grounded this conversation is. We discuss: - Why changing a diaper will be the last task a robot masters - The simulation v. real-world data debate - How multimodal LLMs give robots common sense - Moravec's Paradox + Robot Olympics - Why robots can do long-horizon tasks now - A realistic timeline for robots in our homes I should note that I am an investor in Physical Intelligence -- I made the investment because I believe it is one of the most important companies tackling the problem of robotics. Enjoy! Timestamps: 0:00 Intro 2:39 Defining Physical Intelligence 5:19 The Challenge of Building General Models 6:34 The Stakes and Future of General Purpose Robotics 8:15 Pros and Cons of Humanoid Robots 10:12 Historical Milestones in Robotics Research 15:31 Combining Generative AI and Deep RL 21:24 Moravec's Paradox 25:33 Kitchen Robots 29:30 Simulation vs. Real-World Data 30:48 The Robot Olympics 36:31 The Physiological Reality of Embodiment 38:56 Controversies in the Robotics Community 44:18 What Makes a Great Researcher 48:27 How Businesses Should Prepare for Robotics 54:09 Tracking Progress Through Research Papers 57:02 The Next Step: Mid-Level Reasoning 1:02:00 The Kindest Thing

Patrick OShaughnessy

134,397 views • 5 months ago

This is "Sequential Dexterity", a neural network that controls a robot arm to build legos given a manual 🤖 To do this task, the robot needs to chain together multiple skills (grasping, re-orienting, pushing, etc.) and execute without compounding error. I find some very simple but effective insights in the paper: - Skill chaining over a long sequence is hard. We need a way to "smooth it over". - Forward smoothing: train skill [i] to good performance, run it many times to generate possible "end states", and sample from these states to provide initialization for skill [i+1]. Then start training skill [i+1], and so on. - The problem is that skill [i] is unaware of [i+1], so it may produce bad end states that make life hard for [i+1]. - Backward smoothing: identify what are the good starting states for skill [i+1], then finetune the previous skill [i] towards those states by reinforcement learning. Website: Authors: Yuanpei Chen*, Chen Wang*, Li Fei-Fei, C. Karen Liu Paper: "Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation"
0:46

This is "Sequential Dexterity", a neural network that controls a robot arm to build legos given a manual 🤖 To do this task, the robot needs to chain together multiple skills (grasping, re-orienting, pushing, etc.) and execute without compounding error. I find some very simple but effective insights in the paper: - Skill chaining over a long sequence is hard. We need a way to "smooth it over". - Forward smoothing: train skill [i] to good performance, run it many times to generate possible "end states", and sample from these states to provide initialization for skill [i+1]. Then start training skill [i+1], and so on. - The problem is that skill [i] is unaware of [i+1], so it may produce bad end states that make life hard for [i+1]. - Backward smoothing: identify what are the good starting states for skill [i+1], then finetune the previous skill [i] towards those states by reinforcement learning. Website: Authors: Yuanpei Chen*, Chen Wang*, Li Fei-Fei, C. Karen Liu Paper: "Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation"

Jim Fan

69,064 views • 3 years ago

This is the kind of robotics progress I pay attention to. Toyota introduced Walk Me, a walking wheelchair that moves on mechanical legs instead of wheels. It can climb stairs, handle rough terrain, and adapt to environments that were never designed to be accessible. The point is independence. Getting from A to B without asking the world to be perfectly flat. I have long believed robotics will not be about humanoids. It will be about task shaped machines, designed around real human constraints. Walk Me is a good example of that shift. So here is the question. If robots are shaped by tasks, not appearances, what should we design next? #Robotics #AI #Accessibility #AssistiveTechnology #HumanCenteredDesign #FutureOfWork
0:21

This is the kind of robotics progress I pay attention to. Toyota introduced Walk Me, a walking wheelchair that moves on mechanical legs instead of wheels. It can climb stairs, handle rough terrain, and adapt to environments that were never designed to be accessible. The point is independence. Getting from A to B without asking the world to be perfectly flat. I have long believed robotics will not be about humanoids. It will be about task shaped machines, designed around real human constraints. Walk Me is a good example of that shift. So here is the question. If robots are shaped by tasks, not appearances, what should we design next? #Robotics #AI #Accessibility #AssistiveTechnology #HumanCenteredDesign #FutureOfWork

Pascal Bornet

1,235,999 views • 7 months ago

One of the why questions we’re always asking is how can we help operations employees access inventory more efficiently? We’re taking a big leap forward by introducing DeepFleet, an exciting new AI model that makes our one million+ robot fleet work smarter. Think of DeepFleet as an intelligent traffic management system for our fulfillment centers. Just like smart traffic systems reduce congestion in cities, DeepFleet coordinates our robots’ movements to find optimal paths and reduce bottlenecks. By leveraging our deep expertise in robotics and combining it with AWS tools like SageMaker, we’ve built an AI model that learns from our extensive data on inventory movement to make real-time decisions that improve robot travel time by 10% (which at our scale is pretty significant). What makes this particularly exciting is how AI and robotics create value across the board. For customers, it means faster delivery times and lower costs. For employees, it opens up new opportunities – our robots handle physically demanding tasks and heavy lifting, making our fulfillment centers safer, while our people take on more technical roles. This combination of robotics and generative AI is just getting started, and am looking forward to what we’ll invent next.
0:56

One of the why questions we’re always asking is how can we help operations employees access inventory more efficiently? We’re taking a big leap forward by introducing DeepFleet, an exciting new AI model that makes our one million+ robot fleet work smarter. Think of DeepFleet as an intelligent traffic management system for our fulfillment centers. Just like smart traffic systems reduce congestion in cities, DeepFleet coordinates our robots’ movements to find optimal paths and reduce bottlenecks. By leveraging our deep expertise in robotics and combining it with AWS tools like SageMaker, we’ve built an AI model that learns from our extensive data on inventory movement to make real-time decisions that improve robot travel time by 10% (which at our scale is pretty significant). What makes this particularly exciting is how AI and robotics create value across the board. For customers, it means faster delivery times and lower costs. For employees, it opens up new opportunities – our robots handle physically demanding tasks and heavy lifting, making our fulfillment centers safer, while our people take on more technical roles. This combination of robotics and generative AI is just getting started, and am looking forward to what we’ll invent next.

Andy Jassy

42,327 views • 1 year ago