Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

How can robots reliably place objects in diverse real-world tasks? 🤖🔍 Placement is tough—objects vary in shape and placement modes (such as stacking, hanging, and insertion), making it a challenging problem. We introduce AnyPlace, a two-stage method trained purely on synthetic data to predict diverse placement poses of unseen... show more

Animesh Garg

26,469 subscribers

24,662 Aufrufe • vor 1 Jahr •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Placing objects sounds simple… until robots have to do it. This method makes it simple, fast & reliable. [Github ⬇️] Robotic object placement is tough, especially with stacking, hanging, or insertion. AnyPlace is a new two-stage method that uses only synthetic data and a vision-language model to teach robots where and how to place objects; even in the real world. Why this works ✅ Finds the right spot with help from vision-language models ✅ Handles stacking, insertion, and hanging with no real-world training ✅ Trained on synthetic data using Blender and IsaacSim ✅ Works in the real world without fine-tuning It shows that smart use of simulation and language models can make robotic placement tasks easier, faster, and more reliable. Github: Paper: Thank you for sharing Animesh Garg !

Placing objects sounds simple… until robots have to do it. This method makes it simple, fast & reliable. [Github ⬇️] Robotic object placement is tough, especially with stacking, hanging, or insertion. AnyPlace is a new two-stage method that uses only synthetic data and a vision-language model to teach robots where and how to place objects; even in the real world. Why this works ✅ Finds the right spot with help from vision-language models ✅ Handles stacking, insertion, and hanging with no real-world training ✅ Trained on synthetic data using Blender and IsaacSim ✅ Works in the real world without fine-tuning It shows that smart use of simulation and language models can make robotic placement tasks easier, faster, and more reliable. Github: Paper: Thank you for sharing Animesh Garg !

Ilir Aliu - eu/acc

22,843 Aufrufe • vor 1 Jahr

#Battlefield6's Assault Ladder gadget has two modes. It can be used as a "real" ladder or as a ramp depending on placement angle.

#Battlefield6's Assault Ladder gadget has two modes. It can be used as a "real" ladder or as a ramp depending on placement angle.

Battlefield Bulletin

259,328 Aufrufe • vor 9 Monaten

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot! This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action prediction

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot! This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action prediction

LeRobot

115,194 Aufrufe • vor 7 Monaten

You can actually interact with the world simulator directly in the browser. 🤖 Here is a quick screen recording (8x speed) of me playing with it: real-time action-conditioned video prediction across rigid objects, deformable objects, rope, and object piles. Try it yourself (no install required): Huge kudos to my student Yixuan Wang for making the interactive demo happen!

You can actually interact with the world simulator directly in the browser. 🤖 Here is a quick screen recording (8x speed) of me playing with it: real-time action-conditioned video prediction across rigid objects, deformable objects, rope, and object piles. Try it yourself (no install required): Huge kudos to my student Yixuan Wang for making the interactive demo happen!

Yunzhu Li

41,513 Aufrufe • vor 3 Monaten

A new approach uses diffused orientation fields to help robots transfer skills across curved objects of diverse shapes and sizes, enabling them to peel and slice varied fruits and vegetables. Learn more in Science Robotics:

A new approach uses diffused orientation fields to help robots transfer skills across curved objects of diverse shapes and sizes, enabling them to peel and slice varied fruits and vegetables. Learn more in Science Robotics:

Science Magazine

18,401 Aufrufe • vor 1 Monat

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

Yanjie Ze

75,194 Aufrufe • vor 1 Jahr

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Y Combinator

20,944 Aufrufe • vor 7 Monaten

Imagine robots finally handling cables and wires! 🔌 A new simulation now allows robots to practice handling flexible objects like cables, wires, and hoses in a virtual environment. Handling wires is especially difficult for robots because they are not rigid and behave unpredictably. This simulation captures realistic physics such as bending, tension, and complex movements, helping robots learn more effectively. 📚 Training with these simulated assets helps robots perform better when moving from virtual training to real-world tasks. Applications include electronic device assembly, automotive wire harness installation, smart home setup, and industrial wiring. This approach makes robot training more complete and prepares them for the challenges they will face in real environments.

Imagine robots finally handling cables and wires! 🔌 A new simulation now allows robots to practice handling flexible objects like cables, wires, and hoses in a virtual environment. Handling wires is especially difficult for robots because they are not rigid and behave unpredictably. This simulation captures realistic physics such as bending, tension, and complex movements, helping robots learn more effectively. 📚 Training with these simulated assets helps robots perform better when moving from virtual training to real-world tasks. Applications include electronic device assembly, automotive wire harness installation, smart home setup, and industrial wiring. This approach makes robot training more complete and prepares them for the challenges they will face in real environments.

Lukas Ziegler

29,916 Aufrufe • vor 1 Jahr

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

Jonathan Stephens

17,712 Aufrufe • vor 5 Monaten

Robots struggle with strict action rules…memory and symbols help them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏

Robots struggle with strict action rules…memory and symbols help them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏

Ilir Aliu - eu/acc

10,241 Aufrufe • vor 1 Jahr

How can we responsibly deploy agentic AI robots in real-world environments? As we close the gap between AI reasoning and physical action, developing a new safety framework is key to advancing helpful robots. 🤖

How can we responsibly deploy agentic AI robots in real-world environments? As we close the gap between AI reasoning and physical action, developing a new safety framework is key to advancing helpful robots. 🤖

Google DeepMind

47,798 Aufrufe • vor 8 Monaten

We're currently testing new set design and camera placement AI processes for our portfolio! We're using colored markers to place our "cameras" and characters in specific locations on a specific set! it's similar to preparing a set and scenery for a real film. AI Tools : Midjourney (Set Place 2D) /Freepik : Nano Banana Pro (3D and Camera / character placement) / Grok + Kling (Animation) / Topaz (Upscale) / Characters (See process in quote)

We're currently testing new set design and camera placement AI processes for our portfolio! We're using colored markers to place our "cameras" and characters in specific locations on a specific set! it's similar to preparing a set and scenery for a real film. AI Tools : Midjourney (Set Place 2D) /Freepik : Nano Banana Pro (3D and Camera / character placement) / Grok + Kling (Animation) / Topaz (Upscale) / Characters (See process in quote)

Anima

138,103 Aufrufe • vor 3 Monaten

Check out this Stereo4D paper from Google DeepMind. It's a pretty clever approach to a persistent problem in computer vision -- getting good training data for how things move in 3D. The key insight is using VR180 videos -- those stereo fisheye videos we launched back in 2017 for YouTubeVR. It was always clear that structured stereo datasets would be valuable for computer vision -- and we launched some powerful VR tools with it back in 2017 (link below). But what's the game changer now in 2024 is the scale -- they're providing 110K high quality clips :-) That's the kind of massive, real-world AI dataset that was just a dream back then! They're using it to train this model called DynaDUSt3R that can predict both 3D structure and motion from video frames. Which means it tracks how objects move between frames while simultaneously reconstructing their 3D shape. And given we're dealing with real stereoscopic content, results are notably better than synthetic data, giving you a faithful rendition of the real-world with a diverse set of subject matter. It's one of those through lines when tackling a timeless mission like mapping the world or spatial computing -- VR content created for immersion becoming the foundation for teaching machines to understand how the world moves. Sometimes innovation chains together in unexpected ways! Links to projects below⛓️

Check out this Stereo4D paper from Google DeepMind. It's a pretty clever approach to a persistent problem in computer vision -- getting good training data for how things move in 3D. The key insight is using VR180 videos -- those stereo fisheye videos we launched back in 2017 for YouTubeVR. It was always clear that structured stereo datasets would be valuable for computer vision -- and we launched some powerful VR tools with it back in 2017 (link below). But what's the game changer now in 2024 is the scale -- they're providing 110K high quality clips :-) That's the kind of massive, real-world AI dataset that was just a dream back then! They're using it to train this model called DynaDUSt3R that can predict both 3D structure and motion from video frames. Which means it tracks how objects move between frames while simultaneously reconstructing their 3D shape. And given we're dealing with real stereoscopic content, results are notably better than synthetic data, giving you a faithful rendition of the real-world with a diverse set of subject matter. It's one of those through lines when tackling a timeless mission like mapping the world or spatial computing -- VR content created for immersion becoming the foundation for teaching machines to understand how the world moves. Sometimes innovation chains together in unexpected ways! Links to projects below⛓️

Bilawal Sidhu

67,919 Aufrufe • vor 1 Jahr

Palletizing in the real world! 📦🤖 How do you stack 65 unique SKUs on a pallet when they arrive in random order? Here’s how an on-the-fly algorithm solved it in a real logistics use case with only a single-digit buffer. Every placement was checked for stability, not just for itself, but for every other box it touched. The result? A rock-solid 2.05 m (6.5 ft) pallet. Robotics in logistics keeps improving. Hardware matters, but without smart software your robots won’t know what to do and you’ll waste money and time. Credit: Progressive Robotics

Palletizing in the real world! 📦🤖 How do you stack 65 unique SKUs on a pallet when they arrive in random order? Here’s how an on-the-fly algorithm solved it in a real logistics use case with only a single-digit buffer. Every placement was checked for stability, not just for itself, but for every other box it touched. The result? A rock-solid 2.05 m (6.5 ft) pallet. Robotics in logistics keeps improving. Hardware matters, but without smart software your robots won’t know what to do and you’ll waste money and time. Credit: Progressive Robotics

Ilir Aliu - eu/acc

31,718 Aufrufe • vor 10 Monaten

Explore the latest updates to the Game Animation Sample Project in UE 5.7—now available on Fab, featuring: 🎥 400 new animations 🚶‍♂️ New Movement Modes 🧠 Smart Objects & NPC setups ⛷ Slide mechanic 🦿Foot Placement Control Rig. Get more familiar with the sample project:

Explore the latest updates to the Game Animation Sample Project in UE 5.7—now available on Fab, featuring: 🎥 400 new animations 🚶‍♂️ New Movement Modes 🧠 Smart Objects & NPC setups ⛷ Slide mechanic 🦿Foot Placement Control Rig. Get more familiar with the sample project:

Unreal Engine

66,150 Aufrufe • vor 3 Monaten

#UnityTips Need to spread objects evenly along a transform? Use the L(-1,1) command to linearly distribute them between point 0 and 15. Great for fast level layout, modular environments, and procedural placement workflows. Example from our game #gamedev

#UnityTips Need to spread objects evenly along a transform? Use the L(-1,1) command to linearly distribute them between point 0 and 15. Great for fast level layout, modular environments, and procedural placement workflows. Example from our game #gamedev

Binary Impact

11,055 Aufrufe • vor 28 Tagen

Unreal 5.6 motion tools can now reveal hundreds of objects, such as an entire archviz scene, all in real-time.

Unreal 5.6 motion tools can now reveal hundreds of objects, such as an entire archviz scene, all in real-time.

Jesse Pitela

42,726 Aufrufe • vor 1 Jahr

Introducing MuJoCo 3.0: a major new release of our fast, powerful and open source tool for robotics research. 🤖 📈 GPU & TPU acceleration through #JAX 🖼️ Better simulation of more diverse objects - like clothes, screws, gears and donuts 💡 Find out more:

Introducing MuJoCo 3.0: a major new release of our fast, powerful and open source tool for robotics research. 🤖 📈 GPU & TPU acceleration through #JAX 🖼️ Better simulation of more diverse objects - like clothes, screws, gears and donuts 💡 Find out more:

Google DeepMind

280,498 Aufrufe • vor 2 Jahren

We have been working on blending and this is a simple 1 2 3 Flip It activity for cvc words, real and nonsense words. The cards are numbered to help with the placement. Students worked with a partner.

We have been working on blending and this is a simple 1 2 3 Flip It activity for cvc words, real and nonsense words. The cards are numbered to help with the placement. Students worked with a partner.

Melanie Brethour

66,447 Aufrufe • vor 3 Jahren