Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Demis Hassabis on the next 12 Months: - Full multimodal convergence: Models like Gemini will seamlessly take in and output text, images, audio, and video, with cross-pollination that boosts reasoning + creativity. - Breakthrough visual intelligence: Image models like Nano Banana Pro will produce highly accurate infographics and show... show more

Chubby♨️

111,883 subscribers

89,236 Aufrufe • vor 8 Monaten •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Nano-Banana moment for Video models soon? In an interview two weeks ago, Demis Hassabis talked about the future of image and video models "When Video converges with Language models you will see some interesting combinaton of Capabilites"

Nano-Banana moment for Video models soon? In an interview two weeks ago, Demis Hassabis talked about the future of image and video models "When Video converges with Language models you will see some interesting combinaton of Capabilites"

AiBattle

47,820 Aufrufe • vor 8 Monaten

You can now turn your sources into cinematic video explainers in @NotebookLM 📽️ It uses our most advanced models like Gemini 3, Nano Banana Pro and Veo 3, to animate the visual story behind your sources. Here’s a Cinematic Video Overview explaining... Cinematic Video Overviews ↓

You can now turn your sources into cinematic video explainers in @NotebookLM 📽️ It uses our most advanced models like Gemini 3, Nano Banana Pro and Veo 3, to animate the visual story behind your sources. Here’s a Cinematic Video Overview explaining... Cinematic Video Overviews ↓

Google

102,940 Aufrufe • vor 5 Monaten

Three new open-source models just landed in ComfyUI natively: → Gemma 4 (Google DeepMind) - multimodal LLM handling text, image, audio, and video input with built-in step-by-step reasoning mode → VOID (Netflix) - video object removal that also erases shadows, reflections, and physical interactions caused by the removed subject → BiRefNet - high-res background & object segmentation, one of the most-used segmentation models in the ecosystem Workflows and blog linked below 👇

Three new open-source models just landed in ComfyUI natively: → Gemma 4 (Google DeepMind) - multimodal LLM handling text, image, audio, and video input with built-in step-by-step reasoning mode → VOID (Netflix) - video object removal that also erases shadows, reflections, and physical interactions caused by the removed subject → BiRefNet - high-res background & object segmentation, one of the most-used segmentation models in the ecosystem Workflows and blog linked below 👇

ComfyUI

218,749 Aufrufe • vor 3 Monaten

Demis Hassabis says the Nano Banana Pro system is moving toward AGI for imaging It doesn't just make images, it understands what's in them The goal is to merge this with world models like Genie and SIMA into one system "that might be a candidate for proto-AGI"

Demis Hassabis says the Nano Banana Pro system is moving toward AGI for imaging It doesn't just make images, it understands what's in them The goal is to merge this with world models like Genie and SIMA into one system "that might be a candidate for proto-AGI"

Haider.

50,740 Aufrufe • vor 8 Monaten

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

Salesforce AI Research

13,838 Aufrufe • vor 1 Jahr

Demis Hassabis: AGI will be an intertwined model of Gemini, Genie and something like Nano Banana. Nano Banana shows first climps of AGI "It also has Gemini under the hood, it sort of understands what's going on semantically in those images, so I think that's sort of, it's getting towards a kind of AGIfor imaging" AGI will be the unification of the Genie 3, Nano Banana and Gemini models "And then eventually, we got to kind of converge all of those different projects. And they're intertwined. But we need to, you know, converge them all into one" Google seems to be doing everything right at the moment. With Genie 3, Gemini 3, and Nano Banana, they have all the prerequisites to create a general model that develops a true understanding of the real world.

Demis Hassabis: AGI will be an intertwined model of Gemini, Genie and something like Nano Banana. Nano Banana shows first climps of AGI "It also has Gemini under the hood, it sort of understands what's going on semantically in those images, so I think that's sort of, it's getting towards a kind of AGIfor imaging" AGI will be the unification of the Genie 3, Nano Banana and Gemini models "And then eventually, we got to kind of converge all of those different projects. And they're intertwined. But we need to, you know, converge them all into one" Google seems to be doing everything right at the moment. With Genie 3, Gemini 3, and Nano Banana, they have all the prerequisites to create a general model that develops a true understanding of the real world.

Chubby♨️

66,335 Aufrufe • vor 8 Monaten

Spec ad for a breath strip 😉 Built with invideo Agents & Models - Nano Banana Pro, Kling 2.6 and 70+ more image, video and audio models. All in one workspace. Storyboard. Cast. Generate. Edit. No tab-switching. No multiple workflows. This is what a real AI production studio looks like.

Spec ad for a breath strip 😉 Built with invideo Agents & Models - Nano Banana Pro, Kling 2.6 and 70+ more image, video and audio models. All in one workspace. Storyboard. Cast. Generate. Edit. No tab-switching. No multiple workflows. This is what a real AI production studio looks like.

Invideo

438,539 Aufrufe • vor 7 Monaten

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 Aufrufe • vor 3 Jahren

How does an AI model actually learn to see? 🤖 Learn about the tech behind native multimodality, how models reason over visual data like documents and video, and the future of proactive AI assistants with Logan Kilpatrick and Gemini Model Behavior Product Lead, Ani Baddepudi. ↓ Timestamps: 01:12 Why Gemini is natively multimodal 02:23 The technology behind multimodal models 05:15 Video understanding with Gemini 2.5 09:25 Deciding what to build next 13:23 Building new product experiences with multimodal AI 17:15 The vision for proactive assistants 24:13 Improving video usability with variable FPS and frame tokenization 27:35 What’s next for Gemini’s multimodal development 31:47 Deep dive on Gemini’s document understanding capabilities 37:56 The teamwork and collaboration behind Gemini 40:56 What’s next with model behavior

How does an AI model actually learn to see? 🤖 Learn about the tech behind native multimodality, how models reason over visual data like documents and video, and the future of proactive AI assistants with Logan Kilpatrick and Gemini Model Behavior Product Lead, Ani Baddepudi. ↓ Timestamps: 01:12 Why Gemini is natively multimodal 02:23 The technology behind multimodal models 05:15 Video understanding with Gemini 2.5 09:25 Deciding what to build next 13:23 Building new product experiences with multimodal AI 17:15 The vision for proactive assistants 24:13 Improving video usability with variable FPS and frame tokenization 27:35 What’s next for Gemini’s multimodal development 31:47 Deep dive on Gemini’s document understanding capabilities 37:56 The teamwork and collaboration behind Gemini 40:56 What’s next with model behavior

Google AI

58,703 Aufrufe • vor 1 Jahr

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 Aufrufe • vor 2 Jahren

The latest Qwen 3 VL by Qwen running on iPhone 17 Pro with MLX Qwen 3 VL brings upgraded visual understanding, recognition, and OCR capabilities without sacrificing text performance like previous models The 4B model here is close to Qwen 2.5 VL 72B in many benchmarks

The latest Qwen 3 VL by Qwen running on iPhone 17 Pro with MLX Qwen 3 VL brings upgraded visual understanding, recognition, and OCR capabilities without sacrificing text performance like previous models The 4B model here is close to Qwen 2.5 VL 72B in many benchmarks

Adrien Grondin

109,700 Aufrufe • vor 10 Monaten

Happy GPT-5 Day to all who celebrate. Here's what I expect: It will be agentic-first (this is all but confirmed by OpenAI) It will be heavily multimodal (audio, video, music, tool use, and more) It will be a reasoning-first model as well (math, planning, etc) Directionally, I suspect we're looking at moving towards a convergence of "everything to everything" end-to-end models. Sort of like how Tesla improved FSD by doing neural networks for end-to-end perception-planning-control. Likewise, I suspect that, for the foreseeable future on general purpose models, we're going to converge towards "all in one" models. They will just tend to get smarter and smarter the more modalities we stuff into them (world models, math, image, text, video, audio, etc).

Happy GPT-5 Day to all who celebrate. Here's what I expect: It will be agentic-first (this is all but confirmed by OpenAI) It will be heavily multimodal (audio, video, music, tool use, and more) It will be a reasoning-first model as well (math, planning, etc) Directionally, I suspect we're looking at moving towards a convergence of "everything to everything" end-to-end models. Sort of like how Tesla improved FSD by doing neural networks for end-to-end perception-planning-control. Likewise, I suspect that, for the foreseeable future on general purpose models, we're going to converge towards "all in one" models. They will just tend to get smarter and smarter the more modalities we stuff into them (world models, math, image, text, video, audio, etc).

David Shapiro (L/0)

19,305 Aufrufe • vor 1 Jahr

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️

Xiang Yue

57,699 Aufrufe • vor 1 Jahr

📢 New in Visual Studio: Bring Your Own Model to Chat We’re expanding flexibility in Copilot for Visual Studio. You can now connect API keys from providers like OpenAI, Anthropic, and Google to run Chat on the model of your choice. This update gives you: ✅ Choice – Access the latest models beyond the built-in defaults. ✅ Customization – Select models that fit your infrastructure and requirements. ✅ Control – Manage API usage, limits, and billing directly with your provider. ✅ Flexibility – Switch seamlessly between Copilot’s built-in models and your own. This release is the first step toward a more open and customizable chat experience in Visual Studio, with more providers and capabilities to come. 🔗

📢 New in Visual Studio: Bring Your Own Model to Chat We’re expanding flexibility in Copilot for Visual Studio. You can now connect API keys from providers like OpenAI, Anthropic, and Google to run Chat on the model of your choice. This update gives you: ✅ Choice – Access the latest models beyond the built-in defaults. ✅ Customization – Select models that fit your infrastructure and requirements. ✅ Control – Manage API usage, limits, and billing directly with your provider. ✅ Flexibility – Switch seamlessly between Copilot’s built-in models and your own. This release is the first step toward a more open and customizable chat experience in Visual Studio, with more providers and capabilities to come. 🔗

Visual Studio

11,873 Aufrufe • vor 1 Jahr

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

Andrew Ng

104,371 Aufrufe • vor 2 Jahren

Google DeepMind Developers: How Nano Banana Was Made In this episode, we sat down with Google DeepMind’s Oliver Wang and Nicole Brichtova to discuss how Nano Banana was created, why it became so viral, and what’s in store for the future of image and video editing. 00:00 Intro 02:00 The origin of Nano Banana 06:20 Seeing yourself in AI 11:00 Control & character consistency 17:10 AI in education and visual learning 24:10 2D vs 3D: Debate over world models 31:10 The Japan phenomenon 35:00 From images to video 41:00 Working with artists 47:30 The next era of image models 53:50 Closing thoughts Oliver Wang Nicole Brichtova Guido Appenzeller Yoko

Google DeepMind Developers: How Nano Banana Was Made In this episode, we sat down with Google DeepMind’s Oliver Wang and Nicole Brichtova to discuss how Nano Banana was created, why it became so viral, and what’s in store for the future of image and video editing. 00:00 Intro 02:00 The origin of Nano Banana 06:20 Seeing yourself in AI 11:00 Control & character consistency 17:10 AI in education and visual learning 24:10 2D vs 3D: Debate over world models 31:10 The Japan phenomenon 35:00 From images to video 41:00 Working with artists 47:30 The next era of image models 53:50 Closing thoughts Oliver Wang Nicole Brichtova Guido Appenzeller Yoko

a16z

97,860 Aufrufe • vor 9 Monaten

🚀 Exciting News! We're thrilled to announce the launch of Qwen Chat ( ) – your new go-to Web UI for interacting with Qwen models! 🌟 💬 Chat effortlessly with our flagship model Qwen2.5-Plus , explore vision-language capabilities with Qwen2-VL-Max , and dive into reasoning models like QwQ and QVQ, code with coding expert Qwen2.5-Coder-32B-Instruct, etc. ✨ Key Features: * Choose & compare multiple models in one interface * Upload documents & get answers based on them * Preview mode with HTML support * Upload images for visual understanding 🌟 Coming Soon: Web search, image generation, voice mode, and more! 🔗 Ready to experience the future? Start to chat with Qwen today 💬 Feel free to send us your feedback and suggestions!

🚀 Exciting News! We're thrilled to announce the launch of Qwen Chat ( ) – your new go-to Web UI for interacting with Qwen models! 🌟 💬 Chat effortlessly with our flagship model Qwen2.5-Plus , explore vision-language capabilities with Qwen2-VL-Max , and dive into reasoning models like QwQ and QVQ, code with coding expert Qwen2.5-Coder-32B-Instruct, etc. ✨ Key Features: * Choose & compare multiple models in one interface * Upload documents & get answers based on them * Preview mode with HTML support * Upload images for visual understanding 🌟 Coming Soon: Web search, image generation, voice mode, and more! 🔗 Ready to experience the future? Start to chat with Qwen today 💬 Feel free to send us your feedback and suggestions!

Qwen

215,434 Aufrufe • vor 1 Jahr

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

Wenhu Chen

82,829 Aufrufe • vor 1 Jahr

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,641 Aufrufe • vor 3 Monaten

For an industrial robot built for the rigors of factories and power plants, tidying up a living room may seem like a light day at the office for Spot. Yet, this demonstration represents the promise of AI models in robotics. In this case, Google DeepMind's visual-language model (VLM) Gemini Robotics-ER 1.5 was empowering Spot with embodied reasoning. Go behind the scenes and watch the full demo in our blog:

For an industrial robot built for the rigors of factories and power plants, tidying up a living room may seem like a light day at the office for Spot. Yet, this demonstration represents the promise of AI models in robotics. In this case, Google DeepMind's visual-language model (VLM) Gemini Robotics-ER 1.5 was empowering Spot with embodied reasoning. Go behind the scenes and watch the full demo in our blog:

Boston Dynamics

61,159 Aufrufe • vor 4 Monaten