🤖 Can a single robot policy manipulate diverse tools... without ever seeing them before? Introducing SimToolReal 🔨 : a generalist dexterous manipulation policy that transfers zero-shot sim→real to unseen tools + unseen tasks All videos are 1x speed (60 Hz control) 🧵👇show more

Generalist robot policies need a benchmark that works across... any robot and any policy. 🦾 Introducing RoboLab, a high‑fidelity simulation benchmark built on NVIDIA Isaac and Omniverse to evaluate generalist robot policies in diverse, photoreal, physics‑based environments. Coming soon to the NVIDIA Isaac Lab‑Arena roadmap for large‑scale, robotic policy evaluation. 📖 #NationalRoboticsWeekshow more

Creating a "unified" policy for a robot is uniquely... challenging for two reasons: (a) data is scarce; (b) a single robot can only act sequentially. In PolyTask, we learn a single policy to solve many tasks in a lifelong fashion given *just 1 demonstration* for each task. 🧵show more

How can we more effectively leverage robot data from... different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:show more

Learning bimanual, contact-rich robot manipulation policies that generalize over... diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! 🧵1/nshow more

Skild AI's new S1 robot foundation model helps robots... learn previously unseen tasks from a single video demonstration. 🤖 See how NVIDIA technology supports S1 from training and simulation to real-world deployment. Learn more ➡️show more

How can robots reliably place objects in diverse real-world... tasks? 🤖🔍 Placement is tough—objects vary in shape and placement modes (such as stacking, hanging, and insertion), making it a challenging problem. We introduce AnyPlace, a two-stage method trained purely on synthetic data to predict diverse placement poses of unseen objects for real-world tasks. Read on for more👇show more

How can we leverage diverse human videos to improve... robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.show more

Benchmarking robot policies is broken We ran thousands of... evaluations over 12 different manipulation tasks to evaluate which robot policy you should use.show more

Introducing 2️⃣ new AI systems for robotics: 🤖 ALOHA... Unleashed to perform two-armed manipulation tasks 🦾 DemoStart to control a multi-fingered robotic hand They learned to tackle a range of actions requiring dexterity. Here's how. 🧵show more

🥽 Want to use your new Apple Vision Pro... to control your robot? Want to record how you navigate / manipulate the world to train a policy? I developed an app for VisionOS that can stream your head / wrist / finger movements over WiFi, which you can subscribe on any machines using a simple python library. 🧵[1/6]show more

Robot Utility Models (RUMs) enable basic tasks – door... opening, drawer opening, object reorientation, etc. – at ~90% accuracy without ANY finetuning (i.e. zero-shot) in unseen new environments. Fully open source!!! models, data, code & hw. We think this is super exciting, why?👇 1. Unlocks many practical home utility tasks that often involve these basic tasks as part of an action chain. “Go get me a fork” involves opening the kitchen door and then opening the cutlery drawer. 2. This works well **zero-shot in unseen and new** environments, which is practically a huge deal. Turn the robot on, and get going. 3. The recipe for building a new model is fairly generic, and we think with a bit more refinement this can be a general recipe to build many more Utility models. More details and access 👇show more

🧵Pleased to announce my latest paper: "Act–Observe–Rewrite: Multimodal Coding... Agents as In-Context Policy Learners for Robot Manipulation". What if robots could learn to manipulate objects by rewriting their own code—without demonstrations, training loops, or reward engineering? We demonstrate that this is not only feasible, but you can reproduce the results on a Macbook with a Claude Code subscription in a couple of hours. Let's break it down. 1/8show more

📢#𝗥𝗼𝗯𝗼𝗛𝗶𝘃𝗲 - a unified robot learning framework ✅Designed for... genralizn first robot-learning era ✅Diverse (500 envs, 8 domain) ✅Single flag for Sim<>Real ✅TeleOper Support ✅Multi-(Skill x Task) realworld dataset ✅pip install robohive 🧵👇show more

🤖 Another zero-shot reward model is now in LeRobot:... ROBOMETER. A general-purpose, zero-shot video-language reward model from University of South Carolina, UT Dallas, Massachusetts Institute of Technology (MIT), University of Washington, Ai2, and NVIDIA that predicts frame-level task progress. Trained on 1M+ trajectories from 21 robot embodiments, generalizes zero-shot to unseen tasks, scenes, and robots. 2.4–4.5x better downstream success rates across online RL, offline RL, data filtering, failure detection, and data retrieval for IL. Project: Paper:show more

🤖WILD: ROBOT HANDS ARE NOW A BATTLEGROUND Tesla, BrainCo,... Wuji, and 1X are all racing to build hands with 20-plus degrees of freedom capable of using tools and feeling texture. Some mimic human forearms with tendon-driven fingers while others pack motors directly inside each digit.show more

✨ Any static 3D assets ➡️ 4D dynamic worlds.... Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/nshow more

Long-horizon visual goals remain surprisingly hard for robot manipulation.... We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.show more

Gaussian Splats are really gaining traction recently. Which means... creatives need more tools! Manually editing tools are great - but we want tools that can be used by everyone! Not just 3D or technical artists. a.k.a = automation, sliders and single button presses. Our new 3DGS editing features are done😃 but it wouldn't right to release them without some proper documentation. So we'll be taking care of that, along with a release on Mondayshow more

🤖Why is robot manipulation still an open challenge? This... video shows a kitting task -- packing multiple items into a single product. No robot today can do this autonomously. Big challenges = big opportunities for research and industry. #robotics #manipulation #automationshow more

just made it possible to evaluate generalist robotics models... like Octo at 60-100x real world evaluation speeds via gpu simulation and rendering (~10x faster than original cpu sim code). All videos below are from our open source ManiSkill GPU sim!show more