Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

In their latest video, Boston Dynamics’s AI team explains how they make the Atlas humanoid perceive and interact with the world. Atlas uses an agile perception system to understand both the shape and context of objects in complex environments. Atlas combines 2D and 3D awareness, keypoint-based localization, and an... show more

The Humanoid Hub

115,818 subscribers

50,160 просмотров • 1 год назад •via X (Twitter)

Наука и технологии Образование

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Evin Kamski

Evin Kamski1 год назад

They need to release it doing more tasks, right now moving small packages into other slots is all I have seen, but Atlas’ design is way too heavyweight and will be far too expensive for it to be seen as practical with limited application.

Фото профиля Coral AI News

Coral AI News2 лет назад

Coral AI is the most powerful AI for documents. See the difference yourself:

Фото профиля Corny

Corny1 год назад

Why none of these people on twitter 😭

Фото профиля The Humanoid Hub

The Humanoid Hub1 год назад

coz they have laid low. 🙃 They're here, would love to see them posting more often. @estormyweather @Suddhus @BizaOndrej @czarnowskij

Фото профиля Reborn

Reborn1 год назад

That is incredible.

Фото профиля LoneWordz

LoneWordz1 год назад

Failed enterprise. They're not using neural nets.

Фото профиля Cam Byers

Cam Byers1 год назад

Atlas has the best humanoid hardware design by far.

Фото профиля CIX 🦾

CIX 🦾1 год назад

Their head choice is very sci fi

Фото профиля Taj English

Taj English1 год назад

AhhMazInggg

Фото профиля bimi

bimi1 год назад

Is atlas allowed to work something else? 🤔

Фото профиля Deep Saha

Deep Saha1 год назад

Ever wondered how a humanoid perceives the world? 🤖 The Humanoid Hub dives into Boston Dynamics's AI team's innovative approach with Atlas! Combining 2D and 3D for agile perception. #RobotRevolution #TechInnovation

Похожие видео

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

AI at Meta

163,584 просмотров • 1 год назад

Physical AI for defense and intelligence starts with an accurate model of the world – high-fidelity, geometrically grounded, built for the environments where it actually matters. Today, Niantic Spatial 🌎 is awardable on the Department of War 🇺🇸’s Tradewinds Solutions Marketplace for 3D Reconstruction, our second capability to clear that bar after Visual Positioning, announced earlier this month. Together they deliver two critical layers of the spatial stack: high-fidelity 3D models of contested environments, and precise location within them; a foundation for mission-critical operations that need to visualize and navigate the physical world. The capability is ready. The procurement path is open.

Physical AI for defense and intelligence starts with an accurate model of the world – high-fidelity, geometrically grounded, built for the environments where it actually matters. Today, Niantic Spatial 🌎 is awardable on the Department of War 🇺🇸’s Tradewinds Solutions Marketplace for 3D Reconstruction, our second capability to clear that bar after Visual Positioning, announced earlier this month. Together they deliver two critical layers of the spatial stack: high-fidelity 3D models of contested environments, and precise location within them; a foundation for mission-critical operations that need to visualize and navigate the physical world. The capability is ready. The procurement path is open.

asim ᯅ

14,288 просмотров • 4 месяцев назад

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Over the Reality 🌐

24,045 просмотров • 13 дней назад

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

Xiaolong Wang

43,093 просмотров • 2 лет назад

Introducing Ego2Web from Google DeepMind and UNC Chapel Hill, accepted to #CVPR2026. AI agents can browse the web. But can they act based on what you see? Existing benchmarks focus only on web interaction while ignoring the real world. Ego2Web bridges egocentric video perception and web execution, enabling agents that can see through first-person video, understand real-world context, and take actions on the web grounded in the egocentric video. This opens a path toward AI assistants that operate seamlessly across physical and digital environments. We hope Ego2Web serves as an important step for building more capable, perception-driven agents. 🧵👇

Introducing Ego2Web from Google DeepMind and UNC Chapel Hill, accepted to #CVPR2026. AI agents can browse the web. But can they act based on what you see? Existing benchmarks focus only on web interaction while ignoring the real world. Ego2Web bridges egocentric video perception and web execution, enabling agents that can see through first-person video, understand real-world context, and take actions on the web grounded in the egocentric video. This opens a path toward AI assistants that operate seamlessly across physical and digital environments. We hope Ego2Web serves as an important step for building more capable, perception-driven agents. 🧵👇

Shoubin Yu @ COLM26

46,273 просмотров • 6 месяцев назад

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

Turing Post

13,836 просмотров • 4 месяцев назад

Boston Dynamics has released a new video about humanoid robot hands. "Grippers are one of the most intricate parts of a humanoid robot, packed with actuation and sensing; they need to be durable enough to handle tough jobs, but precise enough to handle fragile items. Our engineers share how the Atlas gripper has evolved as our manipulation capabilities expand and the principles that will guide future iterations."

Boston Dynamics has released a new video about humanoid robot hands. "Grippers are one of the most intricate parts of a humanoid robot, packed with actuation and sensing; they need to be durable enough to handle tough jobs, but precise enough to handle fragile items. Our engineers share how the Atlas gripper has evolved as our manipulation capabilities expand and the principles that will guide future iterations."

Sawyer Merritt

230,895 просмотров • 1 год назад

This is the latest Google research on XR Interaction: World Mouse. Instead of using hand rays or gestures to point at things in the physical space, you move a cursor with a mouse and the system figures out where that cursor should sit in the 3D world. It works in two main ways: - On an object, the cursor follows the surface, so you can move precisely across walls, screens, furniture, or virtual objects. - Between objects, the system creates a smooth bridge through space, so the cursor can travel from one surface to another. That means you can go from a 2D panel to a 3D object, place content on a real wall, manipulate virtual objects, or interact with physical devices through digital proxies. This is the type of news, tools and product updates I share weekly in my newsletter. If you are into 3D XR and AI make sure to check it out (link on top of my profile). Based on “World Mouse: Exploring Interactions with a Cross-Reality Cursor” by Esen K. Tütüncü, Mar Gonzalez-Franco, Khushman Patel, and Eric J. Gonzalez

Gabriele Romagnoli

29,043 просмотров • 6 месяцев назад

Booster T2 is now available. Smart. Powerful. Built to Perform. As a new flagship embodied development platform, Booster T2 delivers up to 2070 TFLOPS of computing power, the highest in its class for bipedal humanoid robots. Combined with a high-output, high-DOF body, it enables sharper motion perception, stronger understanding, and reliable execution in complex scenarios. From real-time perception to rapid response, from agile movement to precise manipulation, Booster T2 combines powerful computing performance and reliable hardware with Booster Studio's integrated development platform, unlocking limitless possibilities for developers. Search "Booster Robotics" to visit the official website and order now.

Booster T2 is now available. Smart. Powerful. Built to Perform. As a new flagship embodied development platform, Booster T2 delivers up to 2070 TFLOPS of computing power, the highest in its class for bipedal humanoid robots. Combined with a high-output, high-DOF body, it enables sharper motion perception, stronger understanding, and reliable execution in complex scenarios. From real-time perception to rapid response, from agile movement to precise manipulation, Booster T2 combines powerful computing performance and reliable hardware with Booster Studio's integrated development platform, unlocking limitless possibilities for developers. Search "Booster Robotics" to visit the official website and order now.

Booster Robotics

1,139,158 просмотров • 3 месяцев назад

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

MrNeRF

42,646 просмотров • 1 год назад

✨Star Atlas - Our First Ever Naabathon Winners 🏆 📢Message From The Team "We are building a platform & we want to enable guilds, engineers, entities to build on top of [Star Atlas] similar to Shopify" Michael Kane, Executive Producer Learn more about each winning project :👇 "Hey everyone, thank you so much for your participation in the first ever Star Atlas Naabathon (hackathon)! We love supporting and enabling co-creation around Star Atlas and we hope you enjoyed participating. Here are the winners:👇 Best Project Award: Atom by meta_arion who won $10,000 USDC, 3,000,000 $ATLAS and an Ogrika Sunpaa! ▪️ Atom has drag and drop user interfaces, and the automations you can do in SAGE are on par, and in some parts better than those built by Star Atlas. Atom won by a unanimous vote. 🔸Galactic Marketplace Award: Atlas Link by harshh.16 and sidh1999 who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ Atlas Link is a AI Chat Bot built on a large language model, indexing all of Star Atlas off chain metadata, on-chain program state, marketplace orders, you can talk to it, communicate to it & strategize and execute on your strategies with it 🔸Defi Award: - StarAtlasLocker by .solwalker who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ Uses Squads multisig to interact with our $ATLAS and $POLIS lockers 🔸Data Award: Star Atlas Crew Dashboard by Aephia Viktor who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic ▪️ 🔸Resources Award: Star Atlas Dispatcher by Teykarat who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ An automation tool for SAGE. Highly complex, highly detailed and tremendous for SAGE. Automate almost everything in SAGE, set it and forget it. 🔸Game Award: Warfront Invasion by visualjc who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! 🔸Star Atlas: Golden Era Award: Eveeye Ship Comparison and Viewer who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ Compare any two assets visually and statistically. The user interface is very impressive, the user experience is very impressive. If you're interested in learning more about these hackathon submissions, be sure to join us for next week's Atlas Brew, where we’ll showcase and discuss the entries!" ⚠️ The Atlas Brew is an official weekly Star Atlas event hosted by Santi, Dom and José in Discord every Wednesday at 10am PT. Topics of the week can range from talking about new releases, team member interviews, theorycrafting and more! :Come join the conversation: #WeAreStarAtlas #Solana #Gamefi #P2E #NFTs #Crypto #StarCitizen #Starfield #Web3gaming

✨Star Atlas - Our First Ever Naabathon Winners 🏆 📢Message From The Team "We are building a platform & we want to enable guilds, engineers, entities to build on top of [Star Atlas] similar to Shopify" Michael Kane, Executive Producer Learn more about each winning project :👇 "Hey everyone, thank you so much for your participation in the first ever Star Atlas Naabathon (hackathon)! We love supporting and enabling co-creation around Star Atlas and we hope you enjoyed participating. Here are the winners:👇 Best Project Award: Atom by meta_arion who won $10,000 USDC, 3,000,000 $ATLAS and an Ogrika Sunpaa! ▪️ Atom has drag and drop user interfaces, and the automations you can do in SAGE are on par, and in some parts better than those built by Star Atlas. Atom won by a unanimous vote. 🔸Galactic Marketplace Award: Atlas Link by harshh.16 and sidh1999 who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ Atlas Link is a AI Chat Bot built on a large language model, indexing all of Star Atlas off chain metadata, on-chain program state, marketplace orders, you can talk to it, communicate to it & strategize and execute on your strategies with it 🔸Defi Award: - StarAtlasLocker by .solwalker who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ Uses Squads multisig to interact with our $ATLAS and $POLIS lockers 🔸Data Award: Star Atlas Crew Dashboard by Aephia Viktor who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic ▪️ 🔸Resources Award: Star Atlas Dispatcher by Teykarat who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ An automation tool for SAGE. Highly complex, highly detailed and tremendous for SAGE. Automate almost everything in SAGE, set it and forget it. 🔸Game Award: Warfront Invasion by visualjc who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! 🔸Star Atlas: Golden Era Award: Eveeye Ship Comparison and Viewer who won $2,500 USDC, 1,500,000 ATLAS and an Ogrika Tursic! ▪️ Compare any two assets visually and statistically. The user interface is very impressive, the user experience is very impressive. If you're interested in learning more about these hackathon submissions, be sure to join us for next week's Atlas Brew, where we’ll showcase and discuss the entries!" ⚠️ The Atlas Brew is an official weekly Star Atlas event hosted by Santi, Dom and José in Discord every Wednesday at 10am PT. Topics of the week can range from talking about new releases, team member interviews, theorycrafting and more! :Come join the conversation: #WeAreStarAtlas #Solana #Gamefi #P2E #NFTs #Crypto #StarCitizen #Starfield #Web3gaming

Siggy

11,867 просмотров • 2 лет назад

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

AI at Meta

310,758 просмотров • 1 год назад

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

The Humanoid Hub

69,472 просмотров • 1 год назад

This week at #CVPR2025, Niantic Spatial is sharing the major strides made toward building a Large Geospatial Model that merges the digital and physical worlds. 🌍🧠 📐MVSAnywhere: Zero-Shot Multi-View Stereo 🎨 Morpheus: Generative 3D Scene Stylization These two research projects reflect a larger ambition: to make AI systems that are spatially aware – able to perceive, interpret, and understand the physical world. 🔗See full blog and GitHub links: Blog post: MVS Anywhere GitHub: Morpheus GitHub: #CVPR2025 #GeospatialAI #NianticSpatial #ComputerVision #3DMapping #AR #GaussianSplatting #DiffusionModels

This week at #CVPR2025, Niantic Spatial is sharing the major strides made toward building a Large Geospatial Model that merges the digital and physical worlds. 🌍🧠 📐MVSAnywhere: Zero-Shot Multi-View Stereo 🎨 Morpheus: Generative 3D Scene Stylization These two research projects reflect a larger ambition: to make AI systems that are spatially aware – able to perceive, interpret, and understand the physical world. 🔗See full blog and GitHub links: Blog post: MVS Anywhere GitHub: Morpheus GitHub: #CVPR2025 #GeospatialAI #NianticSpatial #ComputerVision #3DMapping #AR #GaussianSplatting #DiffusionModels

Niantic Spatial 🌎

11,031 просмотров • 1 год назад

Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python interface, it allows AI agents to learn physical and social reasoning in complex virtual worlds before real world deployment. We're moving from AI that only understands information to AI that can perceive, reason and act inside realistic simulated environments.

Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python interface, it allows AI agents to learn physical and social reasoning in complex virtual worlds before real world deployment. We're moving from AI that only understands information to AI that can perceive, reason and act inside realistic simulated environments.

SciTech Era

47,751 просмотров • 4 месяцев назад

Excited to share ESI-BENCH, a benchmark for Embodied Spatial Intelligence! Most spatial reasoning benchmarks assume an oracle observer: the agent is given the right image, view, or 3D scene. But in the real world, the observer is also an actor. To understand space, agents must decide where to look, how to move, and when to interact, to reveal what is hidden: occlusions, containment, contact, dynamics, and functionality. In many cases, the hard part is not perception itself, but choosing the right action to make informative perception possible. ESI-BENCH tests this perception-action loop. Agents receive an egocentric observation and a spatial question, then must actively gather evidence through perception, locomotion, and manipulationbefore answering. The benchmark spans 10 task categories, 29 subcategories, and 3,081 instances, built in BEHAVIOR-1K across realistic interactive scenes. 🌍Webpage: 💻Code & data: Thanks for collaborators: Jiageng, Han, Manling Li , Leonidas Guibas, Fei-Fei Li , Jiajun Wu , Yejin Choi

Excited to share ESI-BENCH, a benchmark for Embodied Spatial Intelligence! Most spatial reasoning benchmarks assume an oracle observer: the agent is given the right image, view, or 3D scene. But in the real world, the observer is also an actor. To understand space, agents must decide where to look, how to move, and when to interact, to reveal what is hidden: occlusions, containment, contact, dynamics, and functionality. In many cases, the hard part is not perception itself, but choosing the right action to make informative perception possible. ESI-BENCH tests this perception-action loop. Agents receive an egocentric observation and a spatial question, then must actively gather evidence through perception, locomotion, and manipulationbefore answering. The benchmark spans 10 task categories, 29 subcategories, and 3,081 instances, built in BEHAVIOR-1K across realistic interactive scenes. 🌍Webpage: 💻Code & data: Thanks for collaborators: Jiageng, Han, Manling Li , Leonidas Guibas, Fei-Fei Li , Jiajun Wu , Yejin Choi

Yining Hong

53,331 просмотров • 4 месяцев назад

Robots are gaining spatial awareness with Over the Reality. 🤖 Powered by the world’s largest 3D dataset, we enable robots to see, sense, and collaborate in real-world environments. This early test shows how a Unitree Go2 perceives its surroundings through OVER’s VPS — on the right, the real life robot; on the left, its real-time perception of the position inside one of OVER’s 3D maps. Robotic spatial awareness is here, living in 155,000+ mapped locations and expanding every day. One step closer to Physical AI. Stay tuned for our upcoming ROS 2 integration, unlocking a universal framework for robotic perception leveraging OVER's infrastructure!

Robots are gaining spatial awareness with Over the Reality. 🤖 Powered by the world’s largest 3D dataset, we enable robots to see, sense, and collaborate in real-world environments. This early test shows how a Unitree Go2 perceives its surroundings through OVER’s VPS — on the right, the real life robot; on the left, its real-time perception of the position inside one of OVER’s 3D maps. Robotic spatial awareness is here, living in 155,000+ mapped locations and expanding every day. One step closer to Physical AI. Stay tuned for our upcoming ROS 2 integration, unlocking a universal framework for robotic perception leveraging OVER's infrastructure!

Over the Reality 🌐

263,855 просмотров • 1 год назад