How can we leverage diverse human videos to improve... robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.show more

This also shows up in the representations learned by... the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.show more

How can we more effectively leverage robot data from... different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:show more

How can a visuomotor policy learn from internet videos?... We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:show more

𝗥𝗼𝗯𝗼𝘁𝘀 𝗱𝗼𝗻’𝘁 𝗻𝗲𝗲𝗱 𝗺𝗼𝗿𝗲 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻𝘀. 𝗧𝗵𝗲𝘆 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻... 𝗳𝗿𝗼𝗺 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 — 𝗮𝗳𝘁𝗲𝗿 𝘄𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝘀. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: 👉 pretrain on human videos 👉 deploy robot policy 👉 observe failures 👉 reinterpret failures using human priors 👉 improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: 📈 40% → 81% success rate 🏆 Strong improvements over π0.6 RECAP and RISE ✔️ Zero human intervention during post-deployment improvement 🧬 Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same π0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. 📄 Paper: 🌐 Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zürich TU München Microsoft Check out Hanzhi's 🧵 for more detailsshow more

Excited to share a few presentations, demos, and workshop... talks from our group and collaborators at #ICRA2026! We will present recent work on real-to-sim-to-real robot policy evaluation, model-based planning with learned dynamics, and multi-modal manipulation. We will also have a joint live demo between SceniX and Analog Devices, Inc. on real-to-sim-to-real cable manipulation at the ICRA exhibition. This is a small teaser of what we have been building, with more to come soon! If you are at ICRA, please stop by the sessions or the demo booth. Happy to chat about robot learning, simulation, world models, and sim-to-real!show more

Robots have developed the skill to learn by watching... videos: The Visual-Robotics Bridge (VRB) model can teach a robot a new task in roughly 25 minutes using only reference video - without needing human interaction or a simulated environment.show more

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning... Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fangshow more

NVIDIA DROPPED A MOTION DIFFUSION MODEL FOR HUMANOID ROBOTS... trained on 700 hours of mocap data kimodo generates high-quality 3D human and robot motions from text prompts you control it with: → full-body pose keyframes → end-effector positions/rotations → 2D paths and waypoints works on human skeletons and unitree G1 robot plug the outputs directly into mujoco or retarget to other robots using GMR has a web-based interactive demo with a timeline editor. runs locally needs ~17GB VRAM to run inference open source under apache 2.0show more

Teaching robots how to paint! 💅🏼 This painting robot,... for example, mimics human movements with precision to handle repetitive, and tiring tasks. A special device memorizes points in 3D space, which are then sent to the robot's control. The result? A robot that, after a single demonstration, can perform a given action. Keep in mind that it's taught as 'fixed'. 👨🏻‍🔧 That is, it will not be able to react to an anomaly or changing environment. But ultimately, with fewer workers available and many avoiding tough, dirty jobs, robots are helping industries stay efficient and safe. ♻️ RT to help 1 robot find a new workplace!show more

Spent the past few weeks building a fully automated... pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.show more

Train a TensorFlow object detection model – then deploy... it on a robot 🤖 Iulia Feroli (Iulia Feroli) shows how to turn a notebook into a real-time object detection app. This tutorial works for any project – though we demonstrate the deployment on (and assisted by!) the #ReachyMini, an open-source robot from Pollen Robotics. Built with PyCharm + Claude Code. 👉 Watch it in action:show more

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots.... We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:show more

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot!... This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action predictionshow more

30 minutes of video. Robot learns the task. Open-source,... end-to-end. An open-source framework for training robot policies from only 30 minutes of human egocentric videos captured via Meta Aria glasses: Achieving zero-shot transfer to robots without any robot data collection. The method relies on Interaction-Centric Tokens that encode hand-object spatial relationships invariant to embodiment and viewpoint, supplemented by auxiliary objectives like object motion prediction and latent consistency to extract richer supervision signals from the same data. HumanEgo demonstrates strong cross-embodiment, cross-environment performance on bimanual tasks, outperforming baselines like ACT and teleop data while being trainable on a single RTX 4090 GPU. Thanks for sharing, Zhi (Leo) Wang. 📌 Website: Paper: Code: Video: ——- Weekly robotics and AI insights. Subscribe free:show more

Robots don’t just need better brains. They need WAY... more real-world data. 🤖 And collecting high-quality dexterous robot data at scale is one of the hardest problems in physical AI. A fascinating approach is emerging: Wearable human demonstrations + structurally matched dexterous robots. Instead of humans directly teleoperating a robot, Chinese embodied AI startup X Square Robot's TwinDEX system captures the motion, contact, and visual information needed to train the robot — while keeping the data closer to the hardware that will actually execute the task. Early results show promising performance on tool use, fine manipulation, and complex contact-rich tasks. If this approach scales, it could change how we build real-world robotics datasets. The next frontier of physical AI might not be bigger models. It might be better data.show more

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos... with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.show more

𝗗𝗼𝗻'𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗲 𝗿𝗼𝗯𝗼𝘁 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹𝘀. 𝗦𝘁𝗲𝗲𝗿 𝘁𝗵𝗲𝗺 𝘄𝗶𝘁𝗵 𝗵𝘂𝗺𝗮𝗻... 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀 𝗶𝗻𝘀𝘁𝗲𝗮𝗱, 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗯𝗮𝘀𝗲 𝗽𝗼𝗹𝗶𝗰𝘆 Modern VLAs and world-action models can perform impressive manipulation skills, but adapting them reliably to new robots and tasks remains challenging. A natural solution is DAgger-style online imitation learning: deploy the robot, collect human corrections, and update the policy. Yet foundation models are fragile in the low-data regime, fine-tuning on a handful of interventions can improve one behavior while degrading others. Online post-training or reinforcement learning can require costly data collection and exploration, making real-world learning expensive and potentially unsafe. In our new paper, 𝗙𝗹𝗼𝘄𝗗𝗔𝗴𝗴𝗲𝗿, we take a different approach: 𝗜𝗻𝘀𝘁𝗲𝗮𝗱 𝗼𝗳 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹, 𝘄𝗲 𝗹𝗲𝗮𝗿𝗻 𝗵𝗼𝘄 𝘁𝗼 𝘀𝘁𝗲𝗲𝗿 𝗶𝘁 𝗳𝗿𝗼𝗺 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀. The key idea is 𝗮𝗰𝘁𝗶𝗼𝗻 𝗶𝗻𝘃𝗲𝗿𝘀𝗶𝗼𝗻: we map human corrective actions back into the latent noise space of the frozen generative policy. These latent targets train a lightweight controller that adapts the robot while preserving the original model's capabilities. Across simulation and real robots, FlowDAgger: 📈 Learns from only 5–20 human intervention episodes 🏆 Outperforms supervised fine-tuning and latent-space reinforcement learning 🤖 Works across VLAs, diffusion policies, and world-action models ✔️ Provides reliable improvements without modifying the pretrained policy We believe this offers a practical path toward making robot foundation models improve during deployment, learning from the way humans naturally teach: through corrections. 📄 Paper: 🌐 Project: 💻 Code: This project was led by my amazing colleague Michael Murray with help from Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Harshavardhan Reddy Gajarla, Galen Mullins and Andrey Kolobov at Microsoft Research and Maya Cakmak at University of Washingtonshow more

[Most robots react. This one thinks a step ahead.]... Ant Group's Robbyant just published LingBot-VA 2.0 — a video-action foundation model built from scratch for robot control, not fine-tuned from a video generator. The usual approach takes a video generator made for content creation and bolts a robot policy onto it. LingBot-VA 2.0 argues that's the wrong starting point, and pretrains the whole causal stack natively instead. What stands out: → Foresight Reasoning — the robot predicts the next action chunk while executing the current one, then overwrites the imagined frame with the real observation. Prediction and execution stop waiting on each other. → 927 ms → 142 ms per chunk, across four cumulative optimizations. That lifts asynchronous control from 35 Hz to 225 Hz — a 6.5× speedup. → One shared latent space. A semantic visual-action tokenizer puts world states and actions in the same coordinates, so unlabeled web video carries action-relevant signal. → Sparse MoE video stream — 128 experts, top-8 routing. Roughly 2.5B of ~15.3B parameters fire per token. → Few-shot by design — adapts from 10–15 demonstrations, and a human demo video can replace the text instruction entirely. Full breakdown: Paper: Project Page: Robbyant Ant Groupshow more

New startup out of stealth in Cambridge MA: Eka... Robotics. The company is building intelligence for the physical world in its native language: FORCE. Their core solution to mastering fast, reliable adaptive robots: Vision-Force-Action (VFA) model: - sim-only reinforcement learning - no human teleop data - robot practices thousands of hours in a physics-rich simulator (mass, inertia) and comes up with its own solutions, AlphaZero-style. - custom grippers add touch sensing; the model maps pixels + felt force to actions.show more