Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Introducing MazeBench, a 3D open world environment that evaluates long term planning with visual spatial reasoning. It spans hundreds of rooms and puzzles. Today’s best agents cannot progress beyond the initial levels.

💺

5,920 subscribers

236,353 Aufrufe • vor 10 Tagen •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Satya Nadella says the 3 capabilities needed for AI agents are now in place and improving exponentially: a universal interface of multimodality; the neural algebra of reasoning and planning; and long-term memory, rich context and tool use

Satya Nadella says the 3 capabilities needed for AI agents are now in place and improving exponentially: a universal interface of multimodality; the neural algebra of reasoning and planning; and long-term memory, rich context and tool use

Tsarathustra

69,091 Aufrufe • vor 1 Jahr

Scaling 3D scene data is a long-standing challenge in scene understanding, spatial reasoning, and robotics. Since scanning, reconstruction, and labeling are so labor-intensive, data scarcity has remained a major bottleneck. 🛑 To solve this, we propose SceneVerse++: Lifting Unlabeled Internet-level Data for 3D Scene Understanding (CVPR 2026). By reconstructing internet videos and annotating 3D scenes automatically, we’ve created a massive real-world dataset for end-to-end understanding. 🌐📐 SceneVerse++ makes it easy to scale "in-the-wild" 3D scenes toward more capable spatial reasoning systems. This significantly promotes progress in 3D VQA, visual navigation, and broader tasks in Embodied AI and Robotics. 🤖🦾 We are fully open-sourced! Check out the paper, code, and data here: 🌐 Project: 📄 Paper: 📊 Dataset: Code:

Scaling 3D scene data is a long-standing challenge in scene understanding, spatial reasoning, and robotics. Since scanning, reconstruction, and labeling are so labor-intensive, data scarcity has remained a major bottleneck. 🛑 To solve this, we propose SceneVerse++: Lifting Unlabeled Internet-level Data for 3D Scene Understanding (CVPR 2026). By reconstructing internet videos and annotating 3D scenes automatically, we’ve created a massive real-world dataset for end-to-end understanding. 🌐📐 SceneVerse++ makes it easy to scale "in-the-wild" 3D scenes toward more capable spatial reasoning systems. This significantly promotes progress in 3D VQA, visual navigation, and broader tasks in Embodied AI and Robotics. 🤖🦾 We are fully open-sourced! Check out the paper, code, and data here: 🌐 Project: 📄 Paper: 📊 Dataset: Code:

Siyuan Huang

12,612 Aufrufe • vor 3 Monaten

LARP: Language-Agent Role Play for Open-World Games paper page: Language agents have shown impressive problem-solving skills within defined settings and brief timelines. Yet, with the ever-evolving complexities of open-world simulations, there's a pressing need for agents that can flexibly adapt to complex environments and consistently maintain a long-term memory to ensure coherent actions. To bridge the gap between language agents and open-world games, we introduce Language Agent for Role-Playing (LARP), which includes a cognitive architecture that encompasses memory processing and a decision-making assistant, an environment interaction module with a feedback-driven learnable action space, and a postprocessing method that promotes the alignment of various personalities. The LARP framework refines interactions between users and agents, predefined with unique backgrounds and personalities, ultimately enhancing the gaming experience in open-world contexts. Furthermore, it highlights the diverse uses of language models in a range of areas such as entertainment, education, and various simulation scenarios.

LARP: Language-Agent Role Play for Open-World Games paper page: Language agents have shown impressive problem-solving skills within defined settings and brief timelines. Yet, with the ever-evolving complexities of open-world simulations, there's a pressing need for agents that can flexibly adapt to complex environments and consistently maintain a long-term memory to ensure coherent actions. To bridge the gap between language agents and open-world games, we introduce Language Agent for Role-Playing (LARP), which includes a cognitive architecture that encompasses memory processing and a decision-making assistant, an environment interaction module with a feedback-driven learnable action space, and a postprocessing method that promotes the alignment of various personalities. The LARP framework refines interactions between users and agents, predefined with unique backgrounds and personalities, ultimately enhancing the gaming experience in open-world contexts. Furthermore, it highlights the diverse uses of language models in a range of areas such as entertainment, education, and various simulation scenarios.

AK

143,974 Aufrufe • vor 2 Jahren

Elon Musk: The long-term value of SpaceX is immense – it’s just beyond the planning horizon of most investors. WSJ: “Who's going to pay for [the Mars colonization]? Are your investors going to put the money up to do that?” Elon: “I think long-term, the value of it will be incredibly high. It's just beyond the planning horizon of most people or most investors. Obviously, if there's a thriving city on Mars and there's a lot of interplanetary commerce and SpaceX is the primary provider of that, it would be immensely valuable.” Interview with the WSJ, May 2023

Elon Musk: The long-term value of SpaceX is immense – it’s just beyond the planning horizon of most investors. WSJ: “Who's going to pay for [the Mars colonization]? Are your investors going to put the money up to do that?” Elon: “I think long-term, the value of it will be incredibly high. It's just beyond the planning horizon of most people or most investors. Obviously, if there's a thriving city on Mars and there's a lot of interplanetary commerce and SpaceX is the primary provider of that, it would be immensely valuable.” Interview with the WSJ, May 2023

ELON CLIPS

39,678 Aufrufe • vor 2 Monaten

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Jianlan Luo

95,100 Aufrufe • vor 7 Monaten

We’re introducing Cursor 3. It is simpler, more powerful, and built for a world where all code is written by agents, while keeping the depth of a development environment.

We’re introducing Cursor 3. It is simpler, more powerful, and built for a world where all code is written by agents, while keeping the depth of a development environment.

Cursor

2,886,151 Aufrufe • vor 4 Monaten

Introducing General World Models. We believe the next major advancement in AI will come from systems that understand the visual world and its dynamics, which is why we’re starting a new long-term research effort around general world models. Learn more:

Introducing General World Models. We believe the next major advancement in AI will come from systems that understand the visual world and its dynamics, which is why we’re starting a new long-term research effort around general world models. Learn more:

Runway

714,369 Aufrufe • vor 2 Jahren

for physical AI to work in the real world, it needs high-fidelity 3D models of the real world. not just for visualization — for localization, spatial reasoning, and real-to-sim workflows that close the gap between training and deployment. at Niantic Spatial 🌎, we're turning Spexi drone imagery into exactly that — city-scale Gaussian splats grounded in the environments that matter most.

for physical AI to work in the real world, it needs high-fidelity 3D models of the real world. not just for visualization — for localization, spatial reasoning, and real-to-sim workflows that close the gap between training and deployment. at Niantic Spatial 🌎, we're turning Spexi drone imagery into exactly that — city-scale Gaussian splats grounded in the environments that matter most.

asim ᯅ

49,478 Aufrufe • vor 2 Monaten

✨ Added background sounds 🐥🍃🌴 to the World Labs viewer in Interior AI for a more immersive environment This is my new Bali interior style that I then designed living rooms and bathroom with That I then made into a 3d world with [ 🌐 Make 3d world ] that I can walk through with WASD + audio

✨ Added background sounds 🐥🍃🌴 to the World Labs viewer in Interior AI for a more immersive environment This is my new Bali interior style that I then designed living rooms and bathroom with That I then made into a 3d world with [ 🌐 Make 3d world ] that I can walk through with WASD + audio

@levelsio

60,712 Aufrufe • vor 9 Monaten

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n

Fan-Yun Sun

92,584 Aufrufe • vor 1 Jahr

State-of-the-art robot policies often need hundreds of hours of data. What if we needed none? Introducing TiPToP: a manipulation system that zero-shots open-world tasks from pixels and language using vision foundation models and GPU-parallelized Task and Motion Planning (TAMP).

State-of-the-art robot policies often need hundreds of hours of data. What if we needed none? Introducing TiPToP: a manipulation system that zero-shots open-world tasks from pixels and language using vision foundation models and GPU-parallelized Task and Motion Planning (TAMP).

Nishanth Kumar

78,659 Aufrufe • vor 4 Monaten

Introducing Gemini 3.1 Pro 🚀 3.1 Pro represents a major step forward in core reasoning. It scored 77.1% (more than doubling 3 Pro’s score) on ARC-AGI-2, the benchmark that evaluates a model's ability to solve new logic patterns and work through challenges it hasn’t encountered before. This demo illustrates how the model can go beyond the prompt. Instead of rendering a video or static graphic, 3.1 Pro codes a full environment, integrating generative audio and providing UI controls.

Introducing Gemini 3.1 Pro 🚀 3.1 Pro represents a major step forward in core reasoning. It scored 77.1% (more than doubling 3 Pro’s score) on ARC-AGI-2, the benchmark that evaluates a model's ability to solve new logic patterns and work through challenges it hasn’t encountered before. This demo illustrates how the model can go beyond the prompt. Instead of rendering a video or static graphic, 3.1 Pro codes a full environment, integrating generative audio and providing UI controls.

Google AI

3,067,134 Aufrufe • vor 5 Monaten

Physical AI for defense and intelligence starts with an accurate model of the world – high-fidelity, geometrically grounded, built for the environments where it actually matters. Today, Niantic Spatial 🌎 is awardable on the Department of War 🇺🇸’s Tradewinds Solutions Marketplace for 3D Reconstruction, our second capability to clear that bar after Visual Positioning, announced earlier this month. Together they deliver two critical layers of the spatial stack: high-fidelity 3D models of contested environments, and precise location within them; a foundation for mission-critical operations that need to visualize and navigate the physical world. The capability is ready. The procurement path is open.

Physical AI for defense and intelligence starts with an accurate model of the world – high-fidelity, geometrically grounded, built for the environments where it actually matters. Today, Niantic Spatial 🌎 is awardable on the Department of War 🇺🇸’s Tradewinds Solutions Marketplace for 3D Reconstruction, our second capability to clear that bar after Visual Positioning, announced earlier this month. Together they deliver two critical layers of the spatial stack: high-fidelity 3D models of contested environments, and precise location within them; a foundation for mission-critical operations that need to visualize and navigate the physical world. The capability is ready. The procurement path is open.

asim ᯅ

14,225 Aufrufe • vor 2 Monaten

Large-scale Gaussian splats have reached a new level of realism. This is a well-known temple in Bangkok, reconstructed as a high-fidelity 3D environment from 360 captures. At this level, the boundary between video and 3D starts to disappear. But what you’re looking at is not a video. It’s a dense spatial representation of a real place, where geometry, texture, and structure are preserved and made machine-readable. This kind of 3D data can power Visual AI, Robotics navigation, VPS localization, XR experiences, world models, and next-generation spatial computing systems. Built with Over the Reality.

Large-scale Gaussian splats have reached a new level of realism. This is a well-known temple in Bangkok, reconstructed as a high-fidelity 3D environment from 360 captures. At this level, the boundary between video and 3D starts to disappear. But what you’re looking at is not a video. It’s a dense spatial representation of a real place, where geometry, texture, and structure are preserved and made machine-readable. This kind of 3D data can power Visual AI, Robotics navigation, VPS localization, XR experiences, world models, and next-generation spatial computing systems. Built with Over the Reality.

Over the Reality 🌐

347,259 Aufrufe • vor 2 Monaten

Crazy to think Lightning Returns nailed down a Final Fantasy open world and it has been memory holed. Incredible environment full of life, sidequests with tangible rewards, and had its own friction with the world clock that made it unique from every other open world.

Crazy to think Lightning Returns nailed down a Final Fantasy open world and it has been memory holed. Incredible environment full of life, sidequests with tangible rewards, and had its own friction with the world clock that made it unique from every other open world.

Dream's Longest Day

255,991 Aufrufe • vor 3 Monaten

Introducing WorldView 3D, a first-of-its-kind satellite tasking product that delivers up-to-date 3D ground truth anywhere on Earth. From GPS-denied autonomy and intelligence analysis to consumer-grade mapping and disaster response, the world's most important missions increasingly depend on accurate, up-to-date 3D terrain. But for years, keeping 3D maps current at global scale hasn't been possible. WorldView 3D changes that. Built on Vantor’s trusted spatial foundation, our new tasking capability empowers customers to task and receive updated 3D data within 24 hours of image collection. For missions that require greater fidelity, they can also produce high-definition 3D maps anywhere in the world. 3D is the future of spatial intelligence, and Vantor is leading the change. Our trusted spatial foundation already includes 100 million sq km of the world mapped inhighly accurate 3D. Now customers can update the locations that matter most with the speed, scale, and fidelity required for critical missions. Read the full announcement:

Introducing WorldView 3D, a first-of-its-kind satellite tasking product that delivers up-to-date 3D ground truth anywhere on Earth. From GPS-denied autonomy and intelligence analysis to consumer-grade mapping and disaster response, the world's most important missions increasingly depend on accurate, up-to-date 3D terrain. But for years, keeping 3D maps current at global scale hasn't been possible. WorldView 3D changes that. Built on Vantor’s trusted spatial foundation, our new tasking capability empowers customers to task and receive updated 3D data within 24 hours of image collection. For missions that require greater fidelity, they can also produce high-definition 3D maps anywhere in the world. 3D is the future of spatial intelligence, and Vantor is leading the change. Our trusted spatial foundation already includes 100 million sq km of the world mapped inhighly accurate 3D. Now customers can update the locations that matter most with the speed, scale, and fidelity required for critical missions. Read the full announcement:

Vantor

16,030 Aufrufe • vor 1 Monat

Today we’re releasing OpenEQA — the Open-Vocabulary Embodied Question Answering Benchmark. It measures an AI agent’s understanding of physical environments by probing it with open vocabulary questions like “Where did I leave my badge?” More details ➡️ All of today’s state-of-art vision+language models (VLMs) fall well short of human performance. In fact, for questions that require spatial understanding, today’s VLMs are nearly “blind” – access to visual content provides only minor improvements over language-only models. We hope that OpenEQA motivates additional research into helping AI understand and communicate about the world it sees.

Today we’re releasing OpenEQA — the Open-Vocabulary Embodied Question Answering Benchmark. It measures an AI agent’s understanding of physical environments by probing it with open vocabulary questions like “Where did I leave my badge?” More details ➡️ All of today’s state-of-art vision+language models (VLMs) fall well short of human performance. In fact, for questions that require spatial understanding, today’s VLMs are nearly “blind” – access to visual content provides only minor improvements over language-only models. We hope that OpenEQA motivates additional research into helping AI understand and communicate about the world it sees.

AI at Meta

407,109 Aufrufe • vor 2 Jahren

Nick Fuentes GOES OFF on a superchatter that advocates for violence and fully disavows. 🔥 “The power that comes out of a gun is nothing compared to the power that comes with long-term planning.”

Nick Fuentes GOES OFF on a superchatter that advocates for violence and fully disavows. 🔥 “The power that comes out of a gun is nothing compared to the power that comes with long-term planning.”

𝔼𝕥𝕙𝕖𝕣-𝕎𝕒𝕧𝕖

19,655 Aufrufe • vor 1 Tag

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.

AK

141,440 Aufrufe • vor 2 Jahren