Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Here’s an example of Muse Code’s multimodal visual coding capabilities. In this demo, the user inputs a fly-through video of a home into the terminal as an mp4 file. Muse Code interprets the video and produces a visually rich website with booking capabilities.

AI at Meta

831,407 subscribers

21,570 views • 5 days ago •via X (Twitter)

Arts Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Muse Spark 1.1 also excels in perception and multimodal reasoning, inspecting visual and audio inputs, preserving details across long workflows, and acting on them in real execution environments. It shows particular strengths in visual-to-code generation, rich image/video captioning, and agentic computer use. In this demo, using video shot from a smartphone, Muse Spark 1.1 extracts useful photos and reasons about the product to operate a user's browser and make a Facebook Marketplace listing on the user's behalf.

Muse Spark 1.1 also excels in perception and multimodal reasoning, inspecting visual and audio inputs, preserving details across long workflows, and acting on them in real execution environments. It shows particular strengths in visual-to-code generation, rich image/video captioning, and agentic computer use. In this demo, using video shot from a smartphone, Muse Spark 1.1 extracts useful photos and reasons about the product to operate a user's browser and make a Facebook Marketplace listing on the user's behalf.

AI at Meta

73,367 views • 1 month ago

The multimodal and multilingual capabilities of Gemini 3 are quite strong. Here's an example of reading and organizing many different kinds of handwritten recipes and then vibe-coding a bilingual web site.

The multimodal and multilingual capabilities of Gemini 3 are quite strong. Here's an example of reading and organizing many different kinds of handwritten recipes and then vibe-coding a bilingual web site.

Jeff Dean

56,805 views • 8 months ago

With Veo 3 and Flow out in the world, here's a few examples of videos I've created with Veo 3. The first video is an example of the incredible voice/audio capabilities. The second one is a test of doing a longer form video (edited in Premiere). Generated with Veo.

With Veo 3 and Flow out in the world, here's a few examples of videos I've created with Veo 3. The first video is an example of the incredible voice/audio capabilities. The second one is a test of doing a longer form video (edited in Premiere). Generated with Veo.

Martin Nebelong

295,965 views • 1 year ago

VITA Towards Open-Source Interactive Omni Multimodal LLM discuss: The remarkable multimodal capabilities and interactive experience of GPT-4o underscore their necessity in practical applications, yet open-source models rarely excel in both areas. In this paper, we introduce VITA, the first-ever open-source Multimodal Large Language Model (MLLM) adept at simultaneous processing and analysis of Video, Image, Text, and Audio modalities, and meanwhile has an advanced multimodal interactive experience. Starting from Mixtral 8x7B as a language foundation, we expand its Chinese vocabulary followed by bilingual instruction tuning. We further endow the language model with visual and audio capabilities through two-stage multi-task learning of multimodal alignment and instruction tuning. VITA demonstrates robust foundational capabilities of multilingual, vision, and audio understanding, as evidenced by its strong performance across a range of both unimodal and multimodal benchmarks. Beyond foundational capabilities, we have made considerable progress in enhancing the natural multimodal human-computer interaction experience. To the best of our knowledge, we are the first to exploit non-awakening interaction and audio interrupt in MLLM. VITA is the first step for the open-source community to explore the seamless integration of multimodal understanding and interaction. While there is still lots of work to be done on VITA to get close to close-source counterparts, we hope that its role as a pioneer can serve as a cornerstone for subsequent research.

VITA Towards Open-Source Interactive Omni Multimodal LLM discuss: The remarkable multimodal capabilities and interactive experience of GPT-4o underscore their necessity in practical applications, yet open-source models rarely excel in both areas. In this paper, we introduce VITA, the first-ever open-source Multimodal Large Language Model (MLLM) adept at simultaneous processing and analysis of Video, Image, Text, and Audio modalities, and meanwhile has an advanced multimodal interactive experience. Starting from Mixtral 8x7B as a language foundation, we expand its Chinese vocabulary followed by bilingual instruction tuning. We further endow the language model with visual and audio capabilities through two-stage multi-task learning of multimodal alignment and instruction tuning. VITA demonstrates robust foundational capabilities of multilingual, vision, and audio understanding, as evidenced by its strong performance across a range of both unimodal and multimodal benchmarks. Beyond foundational capabilities, we have made considerable progress in enhancing the natural multimodal human-computer interaction experience. To the best of our knowledge, we are the first to exploit non-awakening interaction and audio interrupt in MLLM. VITA is the first step for the open-source community to explore the seamless integration of multimodal understanding and interaction. While there is still lots of work to be done on VITA to get close to close-source counterparts, we hope that its role as a pioneer can serve as a cornerstone for subsequent research.

AK

23,958 views • 2 years ago

Claude Sonnet 4.5 is here and I got to try it early. It has best-in-class capabilities for coding, file creation, and more. I especially love the "Imagine with Claude" feature that lets you create fun apps on the fly. For example, I asked it to "Show me the desktop of a bad PM (left) and a great PM (right)." Here's a video of what it came up with 😀

Claude Sonnet 4.5 is here and I got to try it early. It has best-in-class capabilities for coding, file creation, and more. I especially love the "Imagine with Claude" feature that lets you create fun apps on the fly. For example, I asked it to "Show me the desktop of a bad PM (left) and a great PM (right)." Here's a video of what it came up with 😀

Peter Yang

74,479 views • 10 months ago

The ultimate form of agentic coding will not be the terminal. This is an experiment. An ADE (Agentic Coding Environment), designed from the ground up to work with agentic coding tools. Right now, this only supports Claude Code and runs on a Mac.

The ultimate form of agentic coding will not be the terminal. This is an experiment. An ADE (Agentic Coding Environment), designed from the ground up to work with agentic coding tools. Right now, this only supports Claude Code and runs on a Mac.

Santiago

40,126 views • 5 months ago

co-seeing art with machine: an interactive demo where i browse the visual responses of a multimodal llm to prompts about the painting, from “who’s in here?” and to “what’s unusual about this painting?” the model replies not just with words, but with visual annotations as well.

co-seeing art with machine: an interactive demo where i browse the visual responses of a multimodal llm to prompts about the painting, from “who’s in here?” and to “what’s unusual about this painting?” the model replies not just with words, but with visual annotations as well.

Kat ⊷ the Poet Engineer

37,359 views • 1 year ago

3/ Create a dynamic video using Fliki tool Go to Fliki and paste the text generated by ChatGPT into the tool. This will create a video with a professional avatar presenting the product comparison. Here’s an example of a final video you can create in just 10 minutes:

William | IA

38,077 views • 1 year ago

3/ Create a dynamic video using Fliki AI tool Go to Fliki and paste the text generated by Chat GPT into the tool. This will create a video with a professional avatar presenting the product comparison. Here’s an example of a final video you can create in just 10 minutes:

William | IA

242,168 views • 1 year ago

3/ Create a dynamic video using Fliki AI tool Go to Fliki and paste the text generated by ChatGPT into the tool. This will create a video with a professional avatar presenting the product comparison. Here’s an example of a final video you can create in just 10 minutes:

William | IA

121,965 views • 1 year ago

. @SecDef just signed a letter of intent with the ministry of defense from Saudi Arabia. "This is regarding the modernization and development of the capabilities of the Saudi armed forces through future defense capabilities."

. @SecDef just signed a letter of intent with the ministry of defense from Saudi Arabia. "This is regarding the modernization and development of the capabilities of the Saudi armed forces through future defense capabilities."

DOW Rapid Response

133,065 views • 1 year ago

In this demo, you’ll see Gemini 3 Flash’s frontier-level reasoning and multimodal capabilities on display. The model is able to simultaneously conduct complex geometric calculations while processing complex inputs (video and image). You can play around with the slingshot in Google AI Studio here and share your favorite examples below:

In this demo, you’ll see Gemini 3 Flash’s frontier-level reasoning and multimodal capabilities on display. The model is able to simultaneously conduct complex geometric calculations while processing complex inputs (video and image). You can play around with the slingshot in Google AI Studio here and share your favorite examples below:

Google AI

55,622 views • 7 months ago

Here's the secret behind Cinematic Video Overviews: We put Gemini in the director's chair. This means that Gemini decides the best format (tutorial vs. documentary etc), visual style, and visual capabilities to tell the story of your sources. It then critiques its own footage, refining the visuals and narrative to ensure a seamless, consistent final cut. The result? A bespoke video that converts even the most mundane sources into an engaging, immersive story.

Here's the secret behind Cinematic Video Overviews: We put Gemini in the director's chair. This means that Gemini decides the best format (tutorial vs. documentary etc), visual style, and visual capabilities to tell the story of your sources. It then critiques its own footage, refining the visuals and narrative to ensure a seamless, consistent final cut. The result? A bespoke video that converts even the most mundane sources into an engaging, immersive story.

Gemini Notebook

24,430,131 views • 5 months ago

As one example of 1.5 Pro’s sophisticated multimodal understanding and reasoning capabilities with long context, when given a 44-minute silent film, the model can analyze various plot points and events, and even makes sense of small details you might have missed.

As one example of 1.5 Pro’s sophisticated multimodal understanding and reasoning capabilities with long context, when given a 44-minute silent film, the model can analyze various plot points and events, and even makes sense of small details you might have missed.

Sundar Pichai

348,365 views • 2 years ago

Introducing EvoSkill V1, an open-source toolkit that takes a benchmark and a coding agent, and evolves it into a state-of-the-art specialist in minutes. Here’s all you need to start evolving, starting with your Claude Code ↓

Introducing EvoSkill V1, an open-source toolkit that takes a benchmark and a coding agent, and evolves it into a state-of-the-art specialist in minutes. Here’s all you need to start evolving, starting with your Claude Code ↓

Sentient

30,007 views • 3 months ago

Our new animated explainer premiering today 🎬 Get a visual understanding of AuroryAI and its capabilities. Become an owner, not just a consumer in the AI-powered future! Explore now- AquaGPT-

Our new animated explainer premiering today 🎬 Get a visual understanding of AuroryAI and its capabilities. Become an owner, not just a consumer in the AI-powered future! Explore now- AquaGPT-

Aurory AI

10,816 views • 1 year ago

Kimi K2.5 - Same Prompt - OpenCode vs KIMI CLI vs Claude Code, any differences? 🤔 Create a single-page website for "PHANTOM PROTOCOL" a fictional tactical shooter video game. Design capabilities of this model are out of scale! 🥇 Final results at the end of the video. 🧵

Kimi K2.5 - Same Prompt - OpenCode vs KIMI CLI vs Claude Code, any differences? 🤔 Create a single-page website for "PHANTOM PROTOCOL" a fictional tactical shooter video game. Design capabilities of this model are out of scale! 🥇 Final results at the end of the video. 🧵

Ivan Fioravanti ᯅ

44,652 views • 6 months ago

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 views • 1 year ago

CoD Zombies Remake in HYTOPIA Working on what is probably the first proper demo of a full game mode with This will be included in the developer SDK as an example/template soon. Here's a video of Temptranquil | HYTOPIA and I playing. Still a lot to be done, but fun!

CoD Zombies Remake in HYTOPIA Working on what is probably the first proper demo of a full game mode with This will be included in the developer SDK as an example/template soon. Here's a video of Temptranquil | HYTOPIA and I playing. Still a lot to be done, but fun!

ArkDev | HYTOPIA

14,995 views • 1 year ago