Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🔥In Magma, we talked a lot about spatial/temporal intelligence beyond verbal intelligencen as advocated by Dr. Fei-Fei Li. So how to interpret it? Today I am happy to announce a new demo Magma-Gaming: 👉 Rather than asking LLMs to write game code, we further ask the model to PLAY...

jw2yang4ai's profile picture

Jianwei Yang

3,864 subscribers

17,940 Aufrufe • vor 1 Jahr •via X (Twitter)

GamingWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

8 Kommentare

Profilbild von Jianwei Yang
Jianwei Yangvor 1 Jahr

Magma Project: Magma Code: Magma HF Model: Magma Intro Video:

Profilbild von Jianwei Yang
Jianwei Yangvor 1 Jahr

@alvarobartt @mervenoyann @arankomatsuzaki @NielsRogge

Profilbild von RedDeer.Games
RedDeer.Gamesvor 1 Jahr

We can't spill the beans about the release date of Maki: Paw of Fury, but make no mistake, things are happening! 🫘😎 We remind you that the game is coming to #NintendoSwitch and #PC #Steam and you can play the demo on PC, here ⤵️ >>> Have a great day!

Profilbild von Data & Analytics
Data & Analyticsvor 1 Jahr

@_akhaliq @drfeifei @_akhaliq, exploring spatial and temporal intelligence opens up so many possibilities! Balancing these skills alongside traditional ones could revolutionize our understanding of intelligence. What case studies showcase this best? 🔍 #InnovativeThinking

Profilbild von zaumai
zaumaivor 1 Jahr

@drfeifei Fascinating development! Magma-Gaming's emphasis on spatial and temporal intelligence pushes beyond conventional language-based systems. Any standout scenarios or tests you recommend exploring first?

Profilbild von Oya San
Oya Sanvor 1 Jahr

@drfeifei Absolutely fascinating, @jw2yang4ai! The exploration of spatial and temporal intelligence opens up a universe of possibilities in gaming. I'm excited to see how the Magma-Gaming demo will redefine our interactions and experiences.

Profilbild von Jun (Garvin) Chen
Jun (Garvin) Chenvor 1 Jahr

@drfeifei Brilliant work

Profilbild von scuzzlebot
scuzzlebotvor 1 Jahr

@drfeifei Magma-Gaming highlights spatial intelligence brilliantly—great demonstration of advanced capabilities beyond verbal reasoning! Do you foresee integrating this spatial understanding into more intricate gaming contexts soon?

Ähnliche Videos

"Visual-spatial intelligence is as fundamental as language." Fei-Fei Li says world models are the key to letting AI see the world, reason about it, interact with it, navigate it, and even 'build civilization upon it.' "It's very natural for me that World Labs' north star is to unlock spatial intelligence. The moment to me is right to do it." "We've got these ingredients—we've got compute, we've got a much deeper understanding of data, way deeper than [the ImageNet days]... and we've got some advancement of algorithms."
1:27

"Visual-spatial intelligence is as fundamental as language." Fei-Fei Li says world models are the key to letting AI see the world, reason about it, interact with it, navigate it, and even 'build civilization upon it.' "It's very natural for me that World Labs' north star is to unlock spatial intelligence. The moment to me is right to do it." "We've got these ingredients—we've got compute, we've got a much deeper understanding of data, way deeper than [the ImageNet days]... and we've got some advancement of algorithms."

a16z

31,675 Aufrufe • vor 3 Monaten

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does: "Right now there are three ways of calling world models when it comes to spatial intelligence." "One is what I call a renderer, when the model puts beautiful pixels on the screen." "Another kind of world model is what we call a planner. That is more for machines, more for robots." "The third kind, which I think is the linchpin of the three, is a simulator." "A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on." Fei-Fei Li at Bloomberg Tech live with Emily Chang
2:08

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does: "Right now there are three ways of calling world models when it comes to spatial intelligence." "One is what I call a renderer, when the model puts beautiful pixels on the screen." "Another kind of world model is what we call a planner. That is more for machines, more for robots." "The third kind, which I think is the linchpin of the three, is a simulator." "A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on." Fei-Fei Li at Bloomberg Tech live with Emily Chang

a16z

110,148 Aufrufe • vor 3 Tagen

.Fei-Fei Li, aka The Godmother of AI, discusses her latest startup, World Labs, and why "Spatial" intelligence is a new foundational technology
1:31

.Fei-Fei Li, aka The Godmother of AI, discusses her latest startup, World Labs, and why "Spatial" intelligence is a new foundational technology

Bloomberg TV

34,491 Aufrufe • vor 1 Jahr

Dr. Fei-Fei Li just called out the biggest blind spot in the entire AI industry. We have been building half of human intelligence. And calling it the finish line. Li: “If you look at human intelligence, it pretty much boils down to two buckets.” The first bucket is language. Symbolic reasoning. Communication. The ability to think in words and abstractions. That’s what every major AI lab has spent the last decade building. The second bucket is the one the industry has almost entirely ignored. Li: “We call that in AI spatial intelligence.” How humans and animals perceive, navigate, and interact with the three-dimensional physical world. How we reach for objects. How we move through space. How we build and manipulate physical reality. From painting masterpieces to constructing the pyramids, non-verbal spatial intelligence is what actually shapes the world. Language describes reality. Spatial intelligence acts on it. And the gap between those two things is the gap between a chatbot and a robot. Li: “When this technology is ready, the robotic revolution is gonna start. We’re already seeing that trend.” Every robot is a moving agent. Every moving agent requires spatial intelligence to function in the real world. The humanoid robots being deployed in factories right now are hitting the ceiling of what language models alone can power. Spatial intelligence is the unlock. But Li didn’t stop at robotics. Li: “From a geopolitics point of view, this is part of the technology that goes straight into weapons.” Autonomous drone swarms. Battlefield navigation. Physical target acquisition without human oversight. Every military application of AI that operates in the real world runs on spatial intelligence. The nation that masters the transition from static text to dynamic three-dimensional perception doesn’t just win the software race. It commands the physical battlefield. The AI arms race just broke out of the data center. It’s operating in three dimensions now.
3:08

Dr. Fei-Fei Li just called out the biggest blind spot in the entire AI industry. We have been building half of human intelligence. And calling it the finish line. Li: “If you look at human intelligence, it pretty much boils down to two buckets.” The first bucket is language. Symbolic reasoning. Communication. The ability to think in words and abstractions. That’s what every major AI lab has spent the last decade building. The second bucket is the one the industry has almost entirely ignored. Li: “We call that in AI spatial intelligence.” How humans and animals perceive, navigate, and interact with the three-dimensional physical world. How we reach for objects. How we move through space. How we build and manipulate physical reality. From painting masterpieces to constructing the pyramids, non-verbal spatial intelligence is what actually shapes the world. Language describes reality. Spatial intelligence acts on it. And the gap between those two things is the gap between a chatbot and a robot. Li: “When this technology is ready, the robotic revolution is gonna start. We’re already seeing that trend.” Every robot is a moving agent. Every moving agent requires spatial intelligence to function in the real world. The humanoid robots being deployed in factories right now are hitting the ceiling of what language models alone can power. Spatial intelligence is the unlock. But Li didn’t stop at robotics. Li: “From a geopolitics point of view, this is part of the technology that goes straight into weapons.” Autonomous drone swarms. Battlefield navigation. Physical target acquisition without human oversight. Every military application of AI that operates in the real world runs on spatial intelligence. The nation that masters the transition from static text to dynamic three-dimensional perception doesn’t just win the software race. It commands the physical battlefield. The AI arms race just broke out of the data center. It’s operating in three dimensions now.

Dustin

122,604 Aufrufe • vor 3 Monaten

📁 Fei-Fei Li, computer vision pioneer and Stanford professor, says the next frontier of AI is spatial intelligence. The ability to perceive, reason and act in the physical world. Not just see and talk. But do. Just as vision triggered the Cambrian explosion of life, spatial intelligence could trigger the next explosion of AI.
1:14

📁 Fei-Fei Li, computer vision pioneer and Stanford professor, says the next frontier of AI is spatial intelligence. The ability to perceive, reason and act in the physical world. Not just see and talk. But do. Just as vision triggered the Cambrian explosion of life, spatial intelligence could trigger the next explosion of AI.

Jon Hernandez

24,339 Aufrufe • vor 3 Monaten

The moment is right to push forward into a new frontier for AI — one that is as fundamental as language, says Fei-Fei Li. That frontier is visual spatial intelligence. With Justin Johnson (Justin Johnson), her cofounder at World Labs, and a16z's martin_casado, Fei-Fei explains what unlocking this technology could mean, and why we’re in the midst of a “Cambrian explosion”:
48:09

The moment is right to push forward into a new frontier for AI — one that is as fundamental as language, says Fei-Fei Li. That frontier is visual spatial intelligence. With Justin Johnson (Justin Johnson), her cofounder at World Labs, and a16z's martin_casado, Fei-Fei explains what unlocking this technology could mean, and why we’re in the midst of a “Cambrian explosion”:

a16z

476,523 Aufrufe • vor 1 Jahr

📁 Fei-Fei Li founder of World Labs, says the next leap in AI is not language. Human intelligence does not just speak, it moves, perceives, and acts in the physical world. Spatial intelligence is the real core of intelligence. From text to space, from models to 3D and 4D worlds, from understanding words to interacting with reality. The next chapter is not read, it is inhabited.
1:41

📁 Fei-Fei Li founder of World Labs, says the next leap in AI is not language. Human intelligence does not just speak, it moves, perceives, and acts in the physical world. Spatial intelligence is the real core of intelligence. From text to space, from models to 3D and 4D worlds, from understanding words to interacting with reality. The next chapter is not read, it is inhabited.

Jon Hernandez

20,577 Aufrufe • vor 4 Monaten

a demo of using both the gpt-oss-20b and qwen3-coder-480b in the hyperspace agentic browser. this spatial, AI-native UI is designed as an extension of the spatial space of our brain - which places some things more in recency to us, and adjacent to each other. pinch to zoom.
0:54

a demo of using both the gpt-oss-20b and qwen3-coder-480b in the hyperspace agentic browser. this spatial, AI-native UI is designed as an extension of the spatial space of our brain - which places some things more in recency to us, and adjacent to each other. pinch to zoom.

Varun

180,438 Aufrufe • vor 10 Monaten

On investing in World Modeling and Spatial Intelligence, "It will take time to invest, and it will also hopefully not take the hype..." World Labs' Fei-Fei Li. #BloombergTech Emily Chang ⏯️
1:01

On investing in World Modeling and Spatial Intelligence, "It will take time to invest, and it will also hopefully not take the hype..." World Labs' Fei-Fei Li. #BloombergTech Emily Chang ⏯️

Bloomberg Live

12,843 Aufrufe • vor 7 Tagen

Using AI to write a sonnet is entertaining. Using AI to help perform heart surgery is game-changing. On Possible, "godmother of AI" Fei-Fei Li talks about her team's quest to unlock spatial intelligence.
1:11

Using AI to write a sonnet is entertaining. Using AI to help perform heart surgery is game-changing. On Possible, "godmother of AI" Fei-Fei Li talks about her team's quest to unlock spatial intelligence.

Reid Hoffman

40,120 Aufrufe • vor 1 Jahr

Dr. Fei-Fei Li (Fei-Fei Li), the Godmother of AI, on the origin of computer vision and why spatial intelligence is the next frontier. At AI Startup School in San Francisco. 01:00 - Origins of ImageNet 02:15 - The Dream to Make Machines See 03:28 - A paradigm shift in AI 05:50 - AlexNet and Deep Learning 08:00 - Evolving Computer Vision 12:20 - Building World Labs 13:00 - The next frontier in AI 14:20 - Why Spatial Intelligence Is Harder Than Language 18:40 - Technical Barriers for Vision-Based AI 20:00 - AI is more than LLMs 25:50 - Fei Fei’s Journey 29:30 - Mentoring some of the legends of AI 33:00 - Audience Q&A
44:21

Dr. Fei-Fei Li (Fei-Fei Li), the Godmother of AI, on the origin of computer vision and why spatial intelligence is the next frontier. At AI Startup School in San Francisco. 01:00 - Origins of ImageNet 02:15 - The Dream to Make Machines See 03:28 - A paradigm shift in AI 05:50 - AlexNet and Deep Learning 08:00 - Evolving Computer Vision 12:20 - Building World Labs 13:00 - The next frontier in AI 14:20 - Why Spatial Intelligence Is Harder Than Language 18:40 - Technical Barriers for Vision-Based AI 20:00 - AI is more than LLMs 25:50 - Fei Fei’s Journey 29:30 - Mentoring some of the legends of AI 33:00 - Audience Q&A

Y Combinator

194,386 Aufrufe • vor 11 Monaten

Language models process data as one-dimensional sequences of tokens — ideal for text, but limited for understanding the physical world. In contrast, spatial intelligence embraces a 3D representation, unlocking a deeper understanding of space and time. More from martin_casado's interview with Fei-Fei Li and Justin Johnson:
0:58

Language models process data as one-dimensional sequences of tokens — ideal for text, but limited for understanding the physical world. In contrast, spatial intelligence embraces a 3D representation, unlocking a deeper understanding of space and time. More from martin_casado's interview with Fei-Fei Li and Justin Johnson:

a16z

64,662 Aufrufe • vor 1 Jahr

Excited to share ESI-BENCH, a benchmark for Embodied Spatial Intelligence! Most spatial reasoning benchmarks assume an oracle observer: the agent is given the right image, view, or 3D scene. But in the real world, the observer is also an actor. To understand space, agents must decide where to look, how to move, and when to interact, to reveal what is hidden: occlusions, containment, contact, dynamics, and functionality. In many cases, the hard part is not perception itself, but choosing the right action to make informative perception possible. ESI-BENCH tests this perception-action loop. Agents receive an egocentric observation and a spatial question, then must actively gather evidence through perception, locomotion, and manipulationbefore answering. The benchmark spans 10 task categories, 29 subcategories, and 3,081 instances, built in BEHAVIOR-1K across realistic interactive scenes. 🌍Webpage: 💻Code & data: Thanks for collaborators: Jiageng, Han, Manling Li , Leonidas Guibas, Fei-Fei Li , Jiajun Wu , Yejin Choi
3:36

Excited to share ESI-BENCH, a benchmark for Embodied Spatial Intelligence! Most spatial reasoning benchmarks assume an oracle observer: the agent is given the right image, view, or 3D scene. But in the real world, the observer is also an actor. To understand space, agents must decide where to look, how to move, and when to interact, to reveal what is hidden: occlusions, containment, contact, dynamics, and functionality. In many cases, the hard part is not perception itself, but choosing the right action to make informative perception possible. ESI-BENCH tests this perception-action loop. Agents receive an egocentric observation and a spatial question, then must actively gather evidence through perception, locomotion, and manipulationbefore answering. The benchmark spans 10 task categories, 29 subcategories, and 3,081 instances, built in BEHAVIOR-1K across realistic interactive scenes. 🌍Webpage: 💻Code & data: Thanks for collaborators: Jiageng, Han, Manling Li , Leonidas Guibas, Fei-Fei Li , Jiajun Wu , Yejin Choi

Yining Hong

47,373 Aufrufe • vor 23 Tagen

Spatial intelligence enables the next interface. World Labs co-founders Fei-Fei Li and Justin Johnson say that blending digital and physical worlds simplifies UX and brings agents to the 3D world.
1:16

Spatial intelligence enables the next interface. World Labs co-founders Fei-Fei Li and Justin Johnson say that blending digital and physical worlds simplifies UX and brings agents to the 3D world.

a16z

31,975 Aufrufe • vor 9 Monaten

Here’s a demo of our Agentic Memory system, inspired by how our own brain holds information in a 3D spatial space. This feels, natural. Extending this further, we announced the Agentic Memory Protocol on July 15th in SF - which enables memory to be local, encrypted and available to other agents and apps based on your permission-only. We believe this is the future of memory - not owned by any one app, spatial, and always improving. Aaron Levie Andrej Karpathy
1:03

Here’s a demo of our Agentic Memory system, inspired by how our own brain holds information in a 3D spatial space. This feels, natural. Extending this further, we announced the Agentic Memory Protocol on July 15th in SF - which enables memory to be local, encrypted and available to other agents and apps based on your permission-only. We believe this is the future of memory - not owned by any one app, spatial, and always improving. Aaron Levie Andrej Karpathy

Varun

187,489 Aufrufe • vor 10 Monaten

visionOS 26 quietly introduced a glimpse into the future of how we’ll browse the web: a spatial Reader. Safari can now automatically transform a website’s content into a spatial experience using Spatial Scenes — allowing you to literally look into the rooms of the apartment you’re about to book for your next holiday. It does this by beautifully blending the content with your surroundings and softly tinting your room with the website’s color palette. It's just not possible to capture. I also love that Apple Intelligence lives in a floating window beside the content, ready to summarize the page for you. I can’t wait to see how this evolves into what we’ll one day call the Spatial Web.
0:35

visionOS 26 quietly introduced a glimpse into the future of how we’ll browse the web: a spatial Reader. Safari can now automatically transform a website’s content into a spatial experience using Spatial Scenes — allowing you to literally look into the rooms of the apartment you’re about to book for your next holiday. It does this by beautifully blending the content with your surroundings and softly tinting your room with the website’s color palette. It's just not possible to capture. I also love that Apple Intelligence lives in a floating window beside the content, ready to summarize the page for you. I can’t wait to see how this evolves into what we’ll one day call the Spatial Web.

Phil Traut ᯅ

34,250 Aufrufe • vor 1 Jahr

We present VLM-3R: a Vision-Language Model capable of 3D spatial reasoning from monocular video, grounding visual cues, geometry, and camera motion. ✅ No depth sensor ✅ No pre-built 3D maps ✅ End-to-end spatial + temporal reasoning 🔗 Code & benchmark: #VLM #3DVision #LLMs
0:20

We present VLM-3R: a Vision-Language Model capable of 3D spatial reasoning from monocular video, grounding visual cues, geometry, and camera motion. ✅ No depth sensor ✅ No pre-built 3D maps ✅ End-to-end spatial + temporal reasoning 🔗 Code & benchmark: #VLM #3DVision #LLMs

Zhiwen(Aaron) Fan

14,895 Aufrufe • vor 1 Jahr

Do Vision-Language Models represent space, and how? Spatial terms like "left" or "right" may not be enough to match images with spatial descriptions, as we often overlook the different frames of reference (FoR) used by speakers and listeners. See Figure 1 for examples! Introducing the COnsistent Multilingual Frame Of Reference Test (COMFORT), an evaluation protocol to assess the spatial reasoning capabilities of VLMs. COMFORT includes systematically designed datasets and metrics that evaluate model performance, and their deeper linguistic competence, specifically the spatial knowledge encoded in their internal representations. Find out more in the video teaser! Almost all VLMs prefer the egocentric relative FoR with reflected transform, similar to English. Yet, we reveal significant shortcomings of VLMs: notably, the models (1) exhibit poor robustness and consistency, (2) lack the flexibility to accommodate multiple FoRs, and (3) fail to adhere to language-specific or culture-specific conventions in cross-lingual tests, as English tends to dominate other languages. A shortened version will appear in Pluralistic Alignment Workshop Pluralistic Alignment Workshop #NeurIPS2024. It seems that the ArXiv moderators put it on hold and are eager to give it a thorough read first🤣! So here is the Paper/Code/Data: This collaboration turns out to be amazing, jointly led by Brian Zheyuan Zhang, @Hu_FY_ Jayjun Lee, with so many contributions and insights from Freda Shi, Parisa Kordjamshidi Michigan SLED Lab. With a growing effort to align vision-language models with human cognitive intuitions, we call for more attention to the ambiguous nature and cross-cultural diversity of spatial reasoning!
0:30

Do Vision-Language Models represent space, and how? Spatial terms like "left" or "right" may not be enough to match images with spatial descriptions, as we often overlook the different frames of reference (FoR) used by speakers and listeners. See Figure 1 for examples! Introducing the COnsistent Multilingual Frame Of Reference Test (COMFORT), an evaluation protocol to assess the spatial reasoning capabilities of VLMs. COMFORT includes systematically designed datasets and metrics that evaluate model performance, and their deeper linguistic competence, specifically the spatial knowledge encoded in their internal representations. Find out more in the video teaser! Almost all VLMs prefer the egocentric relative FoR with reflected transform, similar to English. Yet, we reveal significant shortcomings of VLMs: notably, the models (1) exhibit poor robustness and consistency, (2) lack the flexibility to accommodate multiple FoRs, and (3) fail to adhere to language-specific or culture-specific conventions in cross-lingual tests, as English tends to dominate other languages. A shortened version will appear in Pluralistic Alignment Workshop Pluralistic Alignment Workshop #NeurIPS2024. It seems that the ArXiv moderators put it on hold and are eager to give it a thorough read first🤣! So here is the Paper/Code/Data: This collaboration turns out to be amazing, jointly led by Brian Zheyuan Zhang, @Hu_FY_ Jayjun Lee, with so many contributions and insights from Freda Shi, Parisa Kordjamshidi Michigan SLED Lab. With a growing effort to align vision-language models with human cognitive intuitions, we call for more attention to the ambiguous nature and cross-cultural diversity of spatial reasoning!

Martin Ziqiao Ma

35,542 Aufrufe • vor 1 Jahr

Transformers are not the end game. AI still needs a breakthrough I talked to Sanja Fidler, VP of AI Research at NVIDIA, leading company's Spatial Intelligence Lab, and she explains why ↓ And you should definitely watch our full conversation to understand where AI is heading and why physical AI is the next big frontier:
0:35

Transformers are not the end game. AI still needs a breakthrough I talked to Sanja Fidler, VP of AI Research at NVIDIA, leading company's Spatial Intelligence Lab, and she explains why ↓ And you should definitely watch our full conversation to understand where AI is heading and why physical AI is the next big frontier:

Turing Post

77,851 Aufrufe • vor 1 Monat

“Do you know where a sidewalk begins and ends?” — a simple question from Niantic Spatial CEO, Inhi Cho Suh, that gets to the heart of what today’s AI is missing. For robots, this isn’t theoretical. To safely navigate something as everyday as a sidewalk, they need to understand boundaries, context, and change in real time. Learn about how Niantic Spatial is building the missing piece in Physical AI. #NianticSpatial #HumanX #GeospatialAI #Robotics #AI #PhysicalAI
0:39

“Do you know where a sidewalk begins and ends?” — a simple question from Niantic Spatial CEO, Inhi Cho Suh, that gets to the heart of what today’s AI is missing. For robots, this isn’t theoretical. To safely navigate something as everyday as a sidewalk, they need to understand boundaries, context, and change in real time. Learn about how Niantic Spatial is building the missing piece in Physical AI. #NianticSpatial #HumanX #GeospatialAI #Robotics #AI #PhysicalAI

Niantic Spatial 🌎

18,653 Aufrufe • vor 1 Monat