Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

How can robots reliably place objects in diverse real-world tasks? 🤖🔍 Placement is tough—objects vary in shape and placement modes (such as stacking, hanging, and insertion), making it a challenging problem. We introduce AnyPlace, a two-stage method trained purely on synthetic data to predict diverse placement poses of unseen... show more

Animesh Garg

35,022 subscribers

25,378 просмотров • 1 год назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

#Battlefield6's Assault Ladder gadget has two modes. It can be used as a "real" ladder or as a ramp depending on placement angle.

#Battlefield6's Assault Ladder gadget has two modes. It can be used as a "real" ladder or as a ramp depending on placement angle.

Battlefield Bulletin

259,328 просмотров • 10 месяцев назад

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot! This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action prediction

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot! This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action prediction

LeRobot

115,194 просмотров • 8 месяцев назад

You can actually interact with the world simulator directly in the browser. 🤖 Here is a quick screen recording (8x speed) of me playing with it: real-time action-conditioned video prediction across rigid objects, deformable objects, rope, and object piles. Try it yourself (no install required): Huge kudos to my student Yixuan Wang for making the interactive demo happen!

You can actually interact with the world simulator directly in the browser. 🤖 Here is a quick screen recording (8x speed) of me playing with it: real-time action-conditioned video prediction across rigid objects, deformable objects, rope, and object piles. Try it yourself (no install required): Huge kudos to my student Yixuan Wang for making the interactive demo happen!

Yunzhu Li

41,513 просмотров • 4 месяцев назад

A new approach uses diffused orientation fields to help robots transfer skills across curved objects of diverse shapes and sizes, enabling them to peel and slice varied fruits and vegetables. Learn more in Science Robotics:

A new approach uses diffused orientation fields to help robots transfer skills across curved objects of diverse shapes and sizes, enabling them to peel and slice varied fruits and vegetables. Learn more in Science Robotics:

Science Magazine

18,401 просмотров • 2 месяцев назад

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

Yanjie Ze

75,248 просмотров • 1 год назад

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Y Combinator

20,944 просмотров • 7 месяцев назад

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

Jonathan Stephens

17,712 просмотров • 6 месяцев назад

How can we responsibly deploy agentic AI robots in real-world environments? As we close the gap between AI reasoning and physical action, developing a new safety framework is key to advancing helpful robots. 🤖

How can we responsibly deploy agentic AI robots in real-world environments? As we close the gap between AI reasoning and physical action, developing a new safety framework is key to advancing helpful robots. 🤖

Google DeepMind

47,887 просмотров • 9 месяцев назад

Robots struggle with strict action rules…memory and symbols help them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏

Robots struggle with strict action rules…memory and symbols help them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏

Ilir Aliu - eu/acc

10,241 просмотров • 1 год назад

We're currently testing new set design and camera placement AI processes for our portfolio! We're using colored markers to place our "cameras" and characters in specific locations on a specific set! it's similar to preparing a set and scenery for a real film. AI Tools : Midjourney (Set Place 2D) /Freepik : Nano Banana Pro (3D and Camera / character placement) / Grok + Kling (Animation) / Topaz (Upscale) / Characters (See process in quote)

We're currently testing new set design and camera placement AI processes for our portfolio! We're using colored markers to place our "cameras" and characters in specific locations on a specific set! it's similar to preparing a set and scenery for a real film. AI Tools : Midjourney (Set Place 2D) /Freepik : Nano Banana Pro (3D and Camera / character placement) / Grok + Kling (Animation) / Topaz (Upscale) / Characters (See process in quote)

Anima

138,103 просмотров • 4 месяцев назад

Check out this Stereo4D paper from Google DeepMind. It's a pretty clever approach to a persistent problem in computer vision -- getting good training data for how things move in 3D. The key insight is using VR180 videos -- those stereo fisheye videos we launched back in 2017 for YouTubeVR. It was always clear that structured stereo datasets would be valuable for computer vision -- and we launched some powerful VR tools with it back in 2017 (link below). But what's the game changer now in 2024 is the scale -- they're providing 110K high quality clips :-) That's the kind of massive, real-world AI dataset that was just a dream back then! They're using it to train this model called DynaDUSt3R that can predict both 3D structure and motion from video frames. Which means it tracks how objects move between frames while simultaneously reconstructing their 3D shape. And given we're dealing with real stereoscopic content, results are notably better than synthetic data, giving you a faithful rendition of the real-world with a diverse set of subject matter. It's one of those through lines when tackling a timeless mission like mapping the world or spatial computing -- VR content created for immersion becoming the foundation for teaching machines to understand how the world moves. Sometimes innovation chains together in unexpected ways! Links to projects below⛓️

Check out this Stereo4D paper from Google DeepMind. It's a pretty clever approach to a persistent problem in computer vision -- getting good training data for how things move in 3D. The key insight is using VR180 videos -- those stereo fisheye videos we launched back in 2017 for YouTubeVR. It was always clear that structured stereo datasets would be valuable for computer vision -- and we launched some powerful VR tools with it back in 2017 (link below). But what's the game changer now in 2024 is the scale -- they're providing 110K high quality clips :-) That's the kind of massive, real-world AI dataset that was just a dream back then! They're using it to train this model called DynaDUSt3R that can predict both 3D structure and motion from video frames. Which means it tracks how objects move between frames while simultaneously reconstructing their 3D shape. And given we're dealing with real stereoscopic content, results are notably better than synthetic data, giving you a faithful rendition of the real-world with a diverse set of subject matter. It's one of those through lines when tackling a timeless mission like mapping the world or spatial computing -- VR content created for immersion becoming the foundation for teaching machines to understand how the world moves. Sometimes innovation chains together in unexpected ways! Links to projects below⛓️

Bilawal Sidhu

67,919 просмотров • 1 год назад

Palletizing in the real world! 📦🤖 How do you stack 65 unique SKUs on a pallet when they arrive in random order? Here’s how an on-the-fly algorithm solved it in a real logistics use case with only a single-digit buffer. Every placement was checked for stability, not just for itself, but for every other box it touched. The result? A rock-solid 2.05 m (6.5 ft) pallet. Robotics in logistics keeps improving. Hardware matters, but without smart software your robots won’t know what to do and you’ll waste money and time. Credit: Progressive Robotics

Palletizing in the real world! 📦🤖 How do you stack 65 unique SKUs on a pallet when they arrive in random order? Here’s how an on-the-fly algorithm solved it in a real logistics use case with only a single-digit buffer. Every placement was checked for stability, not just for itself, but for every other box it touched. The result? A rock-solid 2.05 m (6.5 ft) pallet. Robotics in logistics keeps improving. Hardware matters, but without smart software your robots won’t know what to do and you’ll waste money and time. Credit: Progressive Robotics

Ilir Aliu - eu/acc

31,718 просмотров • 11 месяцев назад

Explore the latest updates to the Game Animation Sample Project in UE 5.7—now available on Fab, featuring: 🎥 400 new animations 🚶‍♂️ New Movement Modes 🧠 Smart Objects & NPC setups ⛷ Slide mechanic 🦿Foot Placement Control Rig. Get more familiar with the sample project:

Explore the latest updates to the Game Animation Sample Project in UE 5.7—now available on Fab, featuring: 🎥 400 new animations 🚶‍♂️ New Movement Modes 🧠 Smart Objects & NPC setups ⛷ Slide mechanic 🦿Foot Placement Control Rig. Get more familiar with the sample project:

Unreal Engine

66,181 просмотров • 4 месяцев назад

Unreal 5.6 motion tools can now reveal hundreds of objects, such as an entire archviz scene, all in real-time.

Unreal 5.6 motion tools can now reveal hundreds of objects, such as an entire archviz scene, all in real-time.

Jesse Pitela

42,726 просмотров • 1 год назад

#UnityTips Need to spread objects evenly along a transform? Use the L(-1,1) command to linearly distribute them between point 0 and 15. Great for fast level layout, modular environments, and procedural placement workflows. Example from our game #gamedev

#UnityTips Need to spread objects evenly along a transform? Use the L(-1,1) command to linearly distribute them between point 0 and 15. Great for fast level layout, modular environments, and procedural placement workflows. Example from our game #gamedev

Binary Impact

11,090 просмотров • 1 месяц назад

Introducing MuJoCo 3.0: a major new release of our fast, powerful and open source tool for robotics research. 🤖 📈 GPU & TPU acceleration through #JAX 🖼️ Better simulation of more diverse objects - like clothes, screws, gears and donuts 💡 Find out more:

Introducing MuJoCo 3.0: a major new release of our fast, powerful and open source tool for robotics research. 🤖 📈 GPU & TPU acceleration through #JAX 🖼️ Better simulation of more diverse objects - like clothes, screws, gears and donuts 💡 Find out more:

Google DeepMind

280,597 просмотров • 2 лет назад

(1/3) Can we turn text-to-image models into photorealistic 3D generators? ViewDiff (#CVPR2024) produces realistic, multi-view consistent images of real-world 3D objects in authentic surroundings. Website Video How does it work?

(1/3) Can we turn text-to-image models into photorealistic 3D generators? ViewDiff (#CVPR2024) produces realistic, multi-view consistent images of real-world 3D objects in authentic surroundings. Website Video How does it work?

Matthias Niessner

34,753 просмотров • 2 лет назад