How does high-fidelity tactile simulation help robots nail the... last millimeter? We’re releasing VT-Refine, accepted to CoRL: a real-to-sim-to-real visuo-tactile policy using a GPU-parallel tactile sim for our piezoresistive skin FlexiTac. Then fine-tuning a diffusion policy with large-scale RL in simulation. Website: #CoRL2025 #RobotLearning #Sim2Realshow more

Tactile interaction in the wild can unlock fine-grained manipulation!... 🌿🤖✋ We built a portable handheld tactile gripper that enables large-scale visuo-tactile data collection in real-world settings. By pretraining on this data, we bridge vision and touch—allowing robots to: ✅ Perform robust in-hand reorientation ✅ Control contact and force with precision 🔗 Project page: (1/6)show more

1/ 🤲 LeRobot has made low-cost robot learning widely... accessible — but most policies are still blind to contact. Today we release LeFlexiTac: a tactile extension for the LeRobot platform using FlexiTac sensors. Make tactile robot learning as easy as possible. Project page: Code/docs:show more

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical... Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.show more

Want to use tactile sensing but not familiar with... hardware? No worries! Just follow the steps, and you’ll have a high-resolution tactile sensor ready in 30 mins! It’s as simple as making a sandwich! 🥪 🎥 YouTube Tutorial: 🛠️ Open Source & Hardware Guide: 🌐 Project Website: Let’s make robotics more tactile! 🤖 #Robotics #TactileSensing #AI #opensourceshow more

Generalist robot policies need a benchmark that works across... any robot and any policy. 🦾 Introducing RoboLab, a high‑fidelity simulation benchmark built on NVIDIA Isaac and Omniverse to evaluate generalist robot policies in diverse, photoreal, physics‑based environments. Coming soon to the NVIDIA Isaac Lab‑Arena roadmap for large‑scale, robotic policy evaluation. 📖 #NationalRoboticsWeekshow more

Haven't been to a conference in a while, really... excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)show more

Bringing new meaning to the word NeRD at #CoRL2025.... 🤓 Neural Robot Dynamics (NeRD) bridges the sim-to-real gap with a hybrid of neural modeling and analytical simulation, unlocking zero-shot transfer and continuous learning loops for optimal robotics training. Read more & try it in Newton 🤖show more

Meet #HillbotAlpha, the first fully autonomous mobile manipulation robot... trained using sim-to-real technology. Designed in Hillbot’s San Diego headquarters, Hillbot Alpha represents the potential of data synthesis via simulation in robotics. Through a robust strategy that combines a small sample of real-world data with synthetic data, Hillbot Alpha can effectively adapt to evolving task requirements and environments while working safely beside humans. #Hillbot #AGI #EmbodiedAI #AI #Sim2Real #ArtificialIntelligence #Simulation #Simtorealshow more

just made it possible to evaluate generalist robotics models... like Octo at 60-100x real world evaluation speeds via gpu simulation and rendering (~10x faster than original cpu sim code). All videos below are from our open source ManiSkill GPU sim!show more

Sand simulation in Unreal using Niagara GPU particles that... take their velocity from the smoke sim to give it the swirling / spinning motion. All real-time. This will be covered in Niagara Automotive. I just opened the waitlist here:show more

New project! Flow Policy Gradients for Robot Control tldr;... a simple online RL recipe for training and fine-tuning flow policies for robots co-led w/ Hongsuk Benjamin Choi:show more

Trained in Simulation Our robot learns to walk naturally... similar to a human via a high fidelity physics simulator We simulate years of data in only a few hoursshow more

Not to be mistaken with undertaking a horizontal life... pause, I was testing the comfort and durability of the tactile surfaceshow more

What if you could turn a single 360° photo... into a production-ready Isaac Sim environment in minutes? That's exactly what we did here. Using World Labs' Marble and an Insta360 X5 capture (rotating on top), we generated a complete navigable 3D environment and populated it with Lightwheel Sim Ready assets (bottom view). The result? A fully interactive scene in Isaac Sim, ready for sim2real testing,. Navigation, manipulation, or any robotics task you need to validate. What used to take weeks of manual 3D modeling and asset placement now takes minutes. Capture once in the real world, simulate everywhere in your training pipeline. This is the future of robotics development with world models. NVIDIA Robotics NVIDIA Omniverse #Sim2Real #Robotics #Simulationshow more

ManiFeel is a benchmark for contact-rich manipulation tasks. Force-... and tactile-based tasks with occlusions and partially-observable state are going to be crucial for true general purpose robotics. Website:show more

How can a visuomotor policy learn from internet videos?... We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:show more

System identification (sysid) is the process of finding the... physical parameters that make a simulation match reality. If you're training an RL locomotion policy in simulation, the accuracy of your motor model directly affects how well the policy transfers to the real robot. A recent git commit by Kevin Zakka added a sysid toolbox to MuJoCo which automates this process: you provide recorded motor data and a MuJoCo model, and it optimizes the model parameters to minimize the difference between simulated and real trajectories. For my RobStride Dynamics RS02 QDD motors (17 Nm peak, 7.75:1 gear), I built a Rust tool that sends multi-sine torque excitation at 1 kHz and records position/velocity feedback. I then feed this data into MuJoCo's sysid optimizer.show more

Generalist robots need multimodal learning—fusing vision, touch, and body... motion to handle unstructured worlds. 🦾 NVIDIA Isaac Lab is a GPU-accelerated, open-source simulation framework for robot learning that unifies physics, rendering, sensing, and parallel computing to train robot policies at scale. Learn more 🔗 #NVIDIAResearchshow more

As a learning test, I ported part of my... 2D Cloth Physics Simulation to the web using #Rust 🦀 🧪 Try the mini demo: Real-time simulation with Verlet integration, touch/mouse interaction, and web delivery. #gamedev #physics #simulation #verletintegrationshow more

New research from Databricks: LLMs Can Learn to Reason... via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.show more