How can robots reliably place objects in diverse real-world... tasks? 🤖🔍 Placement is tough—objects vary in shape and placement modes (such as stacking, hanging, and insertion), making it a challenging problem. We introduce AnyPlace, a two-stage method trained purely on synthetic data to predict diverse placement poses of unseen objects for real-world tasks. Read on for more👇show more

#Battlefield6's Assault Ladder gadget has two modes. It can... be used as a "real" ladder or as a ramp depending on placement angle.show more

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot!... This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action predictionshow more

You can actually interact with the world simulator directly... in the browser. 🤖 Here is a quick screen recording (8x speed) of me playing with it: real-time action-conditioned video prediction across rigid objects, deformable objects, rope, and object piles. Try it yourself (no install required): Huge kudos to my student Yixuan Wang for making the interactive demo happen!show more

A new approach uses diffused orientation fields to help... robots transfer skills across curved objects of diverse shapes and sizes, enabling them to peel and slice varied fruits and vegetables. Learn more in Science Robotics:show more

We’ve seen humanoid robots walk around for a while,... but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:show more

Spatial AI ( is building large real-world datasets to... teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.show more

You can't 3D reconstruct glass from images... ...WRONG! Thanks... for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AIshow more

How can we responsibly deploy agentic AI robots in... real-world environments? As we close the gap between AI reasoning and physical action, developing a new safety framework is key to advancing helpful robots. 🤖show more

Robots struggle with strict action rules…memory and symbols help... them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏show more

We're currently testing new set design and camera placement... AI processes for our portfolio! We're using colored markers to place our "cameras" and characters in specific locations on a specific set! it's similar to preparing a set and scenery for a real film. AI Tools : Midjourney (Set Place 2D) /Freepik : Nano Banana Pro (3D and Camera / character placement) / Grok + Kling (Animation) / Topaz (Upscale) / Characters (See process in quote)show more

Check out this Stereo4D paper from Google DeepMind. It's... a pretty clever approach to a persistent problem in computer vision -- getting good training data for how things move in 3D. The key insight is using VR180 videos -- those stereo fisheye videos we launched back in 2017 for YouTubeVR. It was always clear that structured stereo datasets would be valuable for computer vision -- and we launched some powerful VR tools with it back in 2017 (link below). But what's the game changer now in 2024 is the scale -- they're providing 110K high quality clips :-) That's the kind of massive, real-world AI dataset that was just a dream back then! They're using it to train this model called DynaDUSt3R that can predict both 3D structure and motion from video frames. Which means it tracks how objects move between frames while simultaneously reconstructing their 3D shape. And given we're dealing with real stereoscopic content, results are notably better than synthetic data, giving you a faithful rendition of the real-world with a diverse set of subject matter. It's one of those through lines when tackling a timeless mission like mapping the world or spatial computing -- VR content created for immersion becoming the foundation for teaching machines to understand how the world moves. Sometimes innovation chains together in unexpected ways! Links to projects below⛓️show more

Palletizing in the real world! 📦🤖 How do you... stack 65 unique SKUs on a pallet when they arrive in random order? Here’s how an on-the-fly algorithm solved it in a real logistics use case with only a single-digit buffer. Every placement was checked for stability, not just for itself, but for every other box it touched. The result? A rock-solid 2.05 m (6.5 ft) pallet. Robotics in logistics keeps improving. Hardware matters, but without smart software your robots won’t know what to do and you’ll waste money and time. Credit: Progressive Roboticsshow more

Explore the latest updates to the Game Animation Sample... Project in UE 5.7—now available on Fab, featuring: 🎥 400 new animations 🚶‍♂️ New Movement Modes 🧠 Smart Objects & NPC setups ⛷ Slide mechanic 🦿Foot Placement Control Rig. Get more familiar with the sample project:show more

Unreal 5.6 motion tools can now reveal hundreds of... objects, such as an entire archviz scene, all in real-time.show more

#UnityTips Need to spread objects evenly along a transform?... Use the L(-1,1) command to linearly distribute them between point 0 and 15. Great for fast level layout, modular environments, and procedural placement workflows. Example from our game #gamedevshow more

Introducing MuJoCo 3.0: a major new release of our... fast, powerful and open source tool for robotics research. 🤖 📈 GPU & TPU acceleration through #JAX 🖼️ Better simulation of more diverse objects - like clothes, screws, gears and donuts 💡 Find out more:show more

(1/3) Can we turn text-to-image models into photorealistic 3D... generators? ViewDiff (#CVPR2024) produces realistic, multi-view consistent images of real-world 3D objects in authentic surroundings. Website Video How does it work?show more