Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Better data for robot hands! 🤲🏼 Embodied reasoning is the bottleneck now. It’s the data we are missing. Specifically, understanding what the robot's hands are doing throughout a task. Perceptron AI new Egocentric offering tracks both hands through an entire video instead of guessing from sampled frames. That matters...

lukas_m_ziegler's profile picture

Lukas Ziegler

56,921 subscribers

17,926 Aufrufe • vor 5 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →
1:23

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

46,829 Aufrufe • vor 7 Monaten

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork
0:20

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,223 Aufrufe • vor 3 Monaten

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:59

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

25,718 Aufrufe • vor 2 Monaten

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.
2:46

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.

TBPN

19,218 Aufrufe • vor 4 Monaten

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER
0:30

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 Aufrufe • vor 1 Jahr

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,334 Aufrufe • vor 1 Monat

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:
1:17

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

Jim Fan

292,967 Aufrufe • vor 4 Monaten

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 Aufrufe • vor 3 Monaten

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:56

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

20,898 Aufrufe • vor 5 Monaten

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

49,920 Aufrufe • vor 6 Monaten

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:
0:46

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,199 Aufrufe • vor 2 Jahren

A simple idea. Let robots collect the data that current foundation models are missing. A robot that gets better by doing real work in the real world. For two weeks in the Stanford East Asia Library, Scanford scanned shelves, helped librarians, and improved the vision language model it depends on. The idea is very simple: Robots do useful work. They gather the real world data foundation models never see online. They fine tune their own model They go back out stronger A full loop. What they found in deployment: ✅ 2103 shelves scanned with multilingual, faded, occluded book spines ✅ 18.7 hours of librarian time saved ✅ Book ID accuracy jumped from 32.0 percent to 71.8 percent ✅ English OCR improved from 24.8 percent to 46.6 percent ✅ Chinese OCR improved from 30.8 percent to 38.0 percent The most interesting part is the shift. Robots do not only consume foundation models. They create the data these models are missing. A clean robot powered data flywheel. Work. Collect. Fine tune. Repeat. Thanks for sharing, Jenn Grannen! If you want the full write up: 📍Website: Paper: —- Weekly robotics and AI insights. Subscribe free:
0:44

A simple idea. Let robots collect the data that current foundation models are missing. A robot that gets better by doing real work in the real world. For two weeks in the Stanford East Asia Library, Scanford scanned shelves, helped librarians, and improved the vision language model it depends on. The idea is very simple: Robots do useful work. They gather the real world data foundation models never see online. They fine tune their own model They go back out stronger A full loop. What they found in deployment: ✅ 2103 shelves scanned with multilingual, faded, occluded book spines ✅ 18.7 hours of librarian time saved ✅ Book ID accuracy jumped from 32.0 percent to 71.8 percent ✅ English OCR improved from 24.8 percent to 46.6 percent ✅ Chinese OCR improved from 30.8 percent to 38.0 percent The most interesting part is the shift. Robots do not only consume foundation models. They create the data these models are missing. A clean robot powered data flywheel. Work. Collect. Fine tune. Repeat. Thanks for sharing, Jenn Grannen! If you want the full write up: 📍Website: Paper: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

44,660 Aufrufe • vor 7 Monaten

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:
0:47

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,812 Aufrufe • vor 8 Monaten

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

59,708 Aufrufe • vor 6 Monaten

The future of farming is autonomous and chemical-free! 🌱 cropr deployed a laser weeding robot working autonomously in chicory fields. AI system identifies weeds from crops. A laser heats the growing point in a fraction of a second. Weed stops growing. No herbicides. No chemicals in the soil. This solves a real problem. Weeds steal water, nutrients, and light. Herbicide resistance is growing. Farmers need alternatives that actually work. The robot learns from every pass. Each field deployment generates data. Each mission improves the next one. The technology already transfers across crops, chicory, carrots, onions coming soon. Better yields. Cleaner fields. Zero chemical residue. That's the future of weed management. 🥕 More info here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:36

The future of farming is autonomous and chemical-free! 🌱 cropr deployed a laser weeding robot working autonomously in chicory fields. AI system identifies weeds from crops. A laser heats the growing point in a fraction of a second. Weed stops growing. No herbicides. No chemicals in the soil. This solves a real problem. Weeds steal water, nutrients, and light. Herbicide resistance is growing. Farmers need alternatives that actually work. The robot learns from every pass. Each field deployment generates data. Each mission improves the next one. The technology already transfers across crops, chicory, carrots, onions coming soon. Better yields. Cleaner fields. Zero chemical residue. That's the future of weed management. 🥕 More info here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

39,443 Aufrufe • vor 15 Tagen

China-based TARS Robotics demonstrated a humanoid robot performing two-handed hand embroidery during a public showcase on December 22, marking a notable step in fine motor control for humanoid systems. The robot threaded a needle and stitched a logo using both hands with sub-millimeter accuracy, working with soft, flexible materials that are difficult for traditional industrial robots to handle due to deformation and variability. According to the company, this capability is enabled by a closed-loop "Data-Al-Physics" system that connects real-world data collection, embodied Al models, and physical execution, reducing the gap between simulation and real deployment. The models are said to be open source when released. Founded in February 2025, TARS Robotics has moved quickly from research to live demonstrations and has raised over $240 million in early funding from the Chinese government, reflecting growing interest in general-purpose humanoid robots for precision and dexterous tasks.
0:58

China-based TARS Robotics demonstrated a humanoid robot performing two-handed hand embroidery during a public showcase on December 22, marking a notable step in fine motor control for humanoid systems. The robot threaded a needle and stitched a logo using both hands with sub-millimeter accuracy, working with soft, flexible materials that are difficult for traditional industrial robots to handle due to deformation and variability. According to the company, this capability is enabled by a closed-loop "Data-Al-Physics" system that connects real-world data collection, embodied Al models, and physical execution, reducing the gap between simulation and real deployment. The models are said to be open source when released. Founded in February 2025, TARS Robotics has moved quickly from research to live demonstrations and has raised over $240 million in early funding from the Chinese government, reflecting growing interest in general-purpose humanoid robots for precision and dexterous tasks.

Brian Roemmele

82,611 Aufrufe • vor 6 Monaten

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast
1:29

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

AI Evolution

15,565 Aufrufe • vor 1 Monat

This robot explores the world like a mouse! 🐁 That's an OG project, guess the year ;) A mouse-inspired bot called WhiskEye travels the world with 24 artificial whiskers and spiking neurons that mimic a nervous system. Recognizing familiar places is essential for engineering applications like robot navigation, where interactions with the real world are key. Combining information from multiple senses helps to make better accuracy and robustness, but challenges like mismatched data and sensor errors complicate this process. Inspired by how animals navigate seamlessly, researchers developed predictive coding, a self-supervised learning method based on the brain's visual cortex model. It combines visual and tactile cues from a robot, outperforming existing methods in recognizing places, even when some sensors fail. What a great project from BristolRoboticsLab and Sheffield Robotics! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:34

This robot explores the world like a mouse! 🐁 That's an OG project, guess the year ;) A mouse-inspired bot called WhiskEye travels the world with 24 artificial whiskers and spiking neurons that mimic a nervous system. Recognizing familiar places is essential for engineering applications like robot navigation, where interactions with the real world are key. Combining information from multiple senses helps to make better accuracy and robustness, but challenges like mismatched data and sensor errors complicate this process. Inspired by how animals navigate seamlessly, researchers developed predictive coding, a self-supervised learning method based on the brain's visual cortex model. It combines visual and tactile cues from a robot, outperforming existing methods in recognizing places, even when some sensors fail. What a great project from BristolRoboticsLab and Sheffield Robotics! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

36,775 Aufrufe • vor 6 Monaten

A policy that teaches robot hands to touch things the way humans do... not just grab and move, but feel and adjust in real time. Robot manipulation research often stops at picking up objects and placing them. CGP goes further: it handles tasks like opening jars, flipping objects in-hand, wiping dishes, and grasping fragile eggs, the kind of dexterous, contact-rich skills that require constant micro-adjustments based on what the fingers are actually feeling. The robot doesn't just see what it's doing; it predicts what contact should feel like at each step, then checks whether reality matches the prediction. If a finger is slipping, the policy knows before the object drops. Works on real robot hands (both 4-finger and 5-finger designs) with tactile sensors embedded in the fingertips Robust to visual distractions! The robot keeps flipping a box correctly even when the camera view is disrupted, because it's grounding decisions in touch, not just vision. Baseline policies without contact grounding fail in predictable ways: slipping mid-task, incomplete motions, loss of grasp, CGP avoids these This is a meaningful step toward robots that can handle the physical world with the kind of reliable, adaptive grip that humans take for granted. Relevant for manufacturing, logistics, assistive robotics, and anywhere fragile or irregular objects need to be handled carefully. Published at RSS 2026, developed with Meta Reality Labs Research. Thanks for sharing, Zhengtong Xu / Zhengtong Xu ——- Weekly robotics and AI insights. Subscribe free:
0:36

A policy that teaches robot hands to touch things the way humans do... not just grab and move, but feel and adjust in real time. Robot manipulation research often stops at picking up objects and placing them. CGP goes further: it handles tasks like opening jars, flipping objects in-hand, wiping dishes, and grasping fragile eggs, the kind of dexterous, contact-rich skills that require constant micro-adjustments based on what the fingers are actually feeling. The robot doesn't just see what it's doing; it predicts what contact should feel like at each step, then checks whether reality matches the prediction. If a finger is slipping, the policy knows before the object drops. Works on real robot hands (both 4-finger and 5-finger designs) with tactile sensors embedded in the fingertips Robust to visual distractions! The robot keeps flipping a box correctly even when the camera view is disrupted, because it's grounding decisions in touch, not just vision. Baseline policies without contact grounding fail in predictable ways: slipping mid-task, incomplete motions, loss of grasp, CGP avoids these This is a meaningful step toward robots that can handle the physical world with the kind of reliable, adaptive grip that humans take for granted. Relevant for manufacturing, logistics, assistive robotics, and anywhere fragile or irregular objects need to be handled carefully. Published at RSS 2026, developed with Meta Reality Labs Research. Thanks for sharing, Zhengtong Xu / Zhengtong Xu ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

12,769 Aufrufe • vor 1 Monat

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.
1:14

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 Aufrufe • vor 4 Monaten