Loading video...

Video Failed to Load

Go Home

How does an AI model actually learn to see? 🤖 Learn about the tech behind native multimodality, how models reason over visual data like documents and video, and the future of proactive AI assistants with Logan Kilpatrick and Gemini Model Behavior Product Lead, Ani Baddepudi. ↓ Timestamps: 01:12 Why...

58,703 views • 1 year ago •via X (Twitter)

11 Comments

Google AI's profile picture
Google AI1 year ago

@AniBaddepudi Watch the full episode here:

Mobile Scanner's profile picture
Mobile Scanner1 year ago

Scan any documents, convert images into text, PDF files, etc. 👍

Fabio Lauria's profile picture
Fabio Lauria1 year ago

@OfficialLoganK @AniBaddepudi AI’s ability to process multimodal data is captivating. It transforms how we interact with technology, bridging gaps between visual perception and reasoning. Excited for the insights from this discussion. #AIFuture

Reji Modiyil's profile picture
Reji Modiyil1 year ago

@OfficialLoganK @AniBaddepudi @GoogleAI, the blending of ai and visual data opens incredible possibilities for innovation.

Cheatify's profile picture
Cheatify1 year ago

@OfficialLoganK @AniBaddepudi @GoogleAI, the evolution of ai vision is fascinating – excited to dive deeper into this topic.

AIMEME's profile picture
AIMEME1 year ago

@OfficialLoganK @AniBaddepudi "AI models learn to see through a combination of advanced technology and continuous learning, paving the way for proactive AI assistants in the future."

Smart AI Stash's profile picture
Smart AI Stash1 year ago

@OfficialLoganK @AniBaddepudi Can’t wait for AI to start critiquing my interior design choices: ‘I can see this is a living room, but why did you choose that couch?’ 😅

^innerly's profile picture
^innerly1 year ago

@OfficialLoganK @AniBaddepudi this ain’t just code, it’s a glimpse at us living next to ai not just staring at screens but actually vibing with the damn thing

Roark Syntax's profile picture
Roark Syntax1 year ago

@OfficialLoganK @AniBaddepudi Neat. #RoarkSyntax

abdelhadi's profile picture
abdelhadi1 year ago

@OfficialLoganK @AniBaddepudi Like so i can come back

Confident Security's profile picture
Confident Security1 year ago

@OfficialLoganK @AniBaddepudi Fascinating topic—just remember that when a model “sees,” it also remembers unless we design for ephemerality. Teaching AI vision should come with equal lessons in how to forget.

Related Videos

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

73,915 views • 1 year ago