How does high-fidelity tactile simulation help robots nail the... last millimeter? We’re releasing VT-Refine, accepted to CoRL: a real-to-sim-to-real visuo-tactile policy using a GPU-parallel tactile sim for our piezoresistive skin FlexiTac. Then fine-tuning a diffusion policy with large-scale RL in simulation. Website: #CoRL2025 #RobotLearning #Sim2Realshow more

1/5 FlexiTac has been accepted to the CoRL 2026... Demo Track, and today we’re releasing FlexiTac-Gen! 🧵 Try it: Over the past two years, we’ve received many requests to customize FlexiTac for different robots. With help from AI agents, we built a tool that makes it easy to design your own sensor and export the files for fabrication. A simple piece of tactile hardware to help move your research forward. See you at CoRL!show more

Tactile interaction in the wild can unlock fine-grained manipulation!... 🌿🤖✋ We built a portable handheld tactile gripper that enables large-scale visuo-tactile data collection in real-world settings. By pretraining on this data, we bridge vision and touch—allowing robots to: ✅ Perform robust in-hand reorientation ✅ Control contact and force with precision 🔗 Project page: (1/6)show more

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical... Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.show more

1/ 🤲 LeRobot has made low-cost robot learning widely... accessible — but most policies are still blind to contact. Today we release LeFlexiTac: a tactile extension for the LeRobot platform using FlexiTac sensors. Make tactile robot learning as easy as possible. Project page: Code/docs:show more

Generalist robot policies need a benchmark that works across... any robot and any policy. 🦾 Introducing RoboLab, a high‑fidelity simulation benchmark built on NVIDIA Isaac and Omniverse to evaluate generalist robot policies in diverse, photoreal, physics‑based environments. Coming soon to the NVIDIA Isaac Lab‑Arena roadmap for large‑scale, robotic policy evaluation. 📖 #NationalRoboticsWeekshow more

Haven't been to a conference in a while, really... excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)show more

Want to use tactile sensing but not familiar with... hardware? No worries! Just follow the steps, and you’ll have a high-resolution tactile sensor ready in 30 mins! It’s as simple as making a sandwich! 🥪 🎥 YouTube Tutorial: 🛠️ Open Source & Hardware Guide: 🌐 Project Website: Let’s make robotics more tactile! 🤖 #Robotics #TactileSensing #AI #opensourceshow more

Introducing Akari: a frontier learned rendering model for robotics... simulation. Akari generates photorealistic rendering for any physics engine to close the sim-to-real gap. Trained in-house, it is the fastest and highest-fidelity learned renderer in any sim platform: 0.046 LPIPS at 0.5s/frame, consistent across 1K+ frames.show more

Bringing new meaning to the word NeRD at #CoRL2025.... 🤓 Neural Robot Dynamics (NeRD) bridges the sim-to-real gap with a hybrid of neural modeling and analytical simulation, unlocking zero-shot transfer and continuous learning loops for optimal robotics training. Read more & try it in Newton 🤖show more

Meet #HillbotAlpha, the first fully autonomous mobile manipulation robot... trained using sim-to-real technology. Designed in Hillbot’s San Diego headquarters, Hillbot Alpha represents the potential of data synthesis via simulation in robotics. Through a robust strategy that combines a small sample of real-world data with synthetic data, Hillbot Alpha can effectively adapt to evolving task requirements and environments while working safely beside humans. #Hillbot #AGI #EmbodiedAI #AI #Sim2Real #ArtificialIntelligence #Simulation #Simtorealshow more

Excited to share a few presentations, demos, and workshop... talks from our group and collaborators at #ICRA2026! We will present recent work on real-to-sim-to-real robot policy evaluation, model-based planning with learned dynamics, and multi-modal manipulation. We will also have a joint live demo between SceniX and Analog Devices, Inc. on real-to-sim-to-real cable manipulation at the ICRA exhibition. This is a small teaser of what we have been building, with more to come soon! If you are at ICRA, please stop by the sessions or the demo booth. Happy to chat about robot learning, simulation, world models, and sim-to-real!show more

Using large-scale GPU parallelized reinforcement learning in Isaac Sim,... we are able to obtain a robust whole-body controller for Taku optimized for precise loco-manipulation.show more

just made it possible to evaluate generalist robotics models... like Octo at 60-100x real world evaluation speeds via gpu simulation and rendering (~10x faster than original cpu sim code). All videos below are from our open source ManiSkill GPU sim!show more

Sand simulation in Unreal using Niagara GPU particles that... take their velocity from the smoke sim to give it the swirling / spinning motion. All real-time. This will be covered in Niagara Automotive. I just opened the waitlist here:show more

New project! Flow Policy Gradients for Robot Control tldr;... a simple online RL recipe for training and fine-tuning flow policies for robots co-led w/ Hongsuk Benjamin Choi:show more

Trained in Simulation Our robot learns to walk naturally... similar to a human via a high fidelity physics simulator We simulate years of data in only a few hoursshow more

Sharpa Robotics Sharpa observed that teleoperating a dual hand... robot offering a combined 63 DoF is a nightmare, even for an experienced operator. They offer a two fold solution: - a VLA that handles "in hand rotation" (like the apple in video below), that the operator can trigger using a foot pedal. - a Mixture-of-Dexterous-Experts that fuses tactile sensing data. Interestingly enough, the first model handles the apple peeling end-to-end, tactile data does not help here. However, it greatly enhances the insertion and grasping of the apple. This research makes use of the SharpaWave 22-DoF dexterous hand, with force and tactile feedback on all ten fingertips. I find it interesting that is now possible to delegate complex low-level finger coordination tasks to RL-trained atomic skills. Optimizing such steps that are difficult to reproduce in teleoperation significantly helps overcoming the data acquisition bottleneck.show more

What if you could turn a single 360° photo... into a production-ready Isaac Sim environment in minutes? That's exactly what we did here. Using World Labs' Marble and an Insta360 X5 capture (rotating on top), we generated a complete navigable 3D environment and populated it with Lightwheel Sim Ready assets (bottom view). The result? A fully interactive scene in Isaac Sim, ready for sim2real testing,. Navigation, manipulation, or any robotics task you need to validate. What used to take weeks of manual 3D modeling and asset placement now takes minutes. Capture once in the real world, simulate everywhere in your training pipeline. This is the future of robotics development with world models. NVIDIA Robotics NVIDIA Omniverse #Sim2Real #Robotics #Simulationshow more

adding sounds to animations makes them just a tad... bit more tactile and memorable.show more

System identification (sysid) is the process of finding the... physical parameters that make a simulation match reality. If you're training an RL locomotion policy in simulation, the accuracy of your motor model directly affects how well the policy transfers to the real robot. A recent git commit by Kevin Zakka added a sysid toolbox to MuJoCo which automates this process: you provide recorded motor data and a MuJoCo model, and it optimizes the model parameters to minimize the difference between simulated and real trajectories. For my RobStride Dynamics RS02 QDD motors (17 Nm peak, 7.75:1 gear), I built a Rust tool that sends multi-sine torque excitation at 1 kHz and records position/velocity feedback. I then feed this data into MuJoCo's sysid optimizer.show more