Loading video...

Video Failed to Load

Go Home

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata...

108,622 views • 2 months ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

I just built an AI-powered creative search engine with Gemini Embedding 2 + Claude Code 🤯 Drop in your UGC clips, product shots, and ad variations — then search through everything in plain English. "Show me all the unboxing clips." "Find product shots with natural lighting." "Which creator talked about sensitive skin?" All inside Claude Code. Perfect for DTC brands and agencies sitting on hundreds of creative files they can never find when they actually need them. If you're digging through a folder of random file names, scrubbing through raw footage to find that one clip, and relying on memory to track down what's already been shot... This system eliminates the entire loop: → Drop your videos, images, and docs into a project folder → One prompt to Claude Code — it builds the entire search app for you → Google's new Gemini Embedding 2 model actually watches your videos and looks at your images → It understands what's inside each file — not just the file name → Search in plain English and get back the actual assets with confidence scores No scrolling through folders. No relying on file names to find anything. No re-shooting footage you already have. What you get: → A searchable library of every creative asset your brand has ever produced → Natural language search across video, images, and documents at the same time → Results that show the actual files inline — play videos, view images, read docs → A system that gets smarter every time you add richer descriptions to your assets One free API key. No monthly subscriptions. Runs on your machine. I put together a full playbook with the exact build prompt, the setup process, and DTC/agency use cases to get this running in under 30 minutes. Want the full playbook? > Like this post > Comment "SEARCH" And I'll send it over (must be following so I can DM)

Mike Futia

12,239 views • 4 months ago

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 views • 1 year ago

⚡️We are excited to announce that our new no-code Enterprise Platform is NOW available in private beta! As RAG apps advance from prototype to production we’ve been overwhelmed by requests for an enterprise grade solution to provide these applications with the data they need. Designed to make it easy to get your data #RAGready, our Platform can preprocess more than 25 file types and soon will be fully #multimodal, also able to ingest audio, video and image files. We ship with a baseline suite of source connectors, including Amazon Web Services S3, Microsoft Azure Blob Storage, OneDrive, SFTP, Databricks Delta Table, Google Drive, Salesforce, Elastic, OpenSearch, and Google Cloud storage with many more fast following. Platform transforms your documents into a standardized JSON schema, broken down into semantically coherent elements allowing you to reconstruct your document in the manner most useful to you. Want only the narrative text but not the headers and footers? This is entirely configurable through the UI. Additionally, we generate more than 30 types of metadata for each element to make it easy to curate the data being written downstream and to support metadata filtering during retrieval. Smart chunking and the ability to choose from a range of embedding models are in from launch, delivering a turnkey solution for chunk and embedding experimentation. As for destination connectors, we've got that covered too, with Amazon Web Services S3, Pinecone, Chroma , Weaviate AI Database, Google Cloud storage, MongoDB, Microsoft Azure cognitive search, PostgreSQL, Elastic, OpenSearch, and Databricks Delta Table. And of course, all of this can be scheduled to keep your data continuously hydrated. The private-beta is live today! Sign-up to get access and come build the future of LLM data foundations with us: 🚀 #ETLforLLMs #AI #DataPreprocessing #DataScience #DataTransformation #LLMs #ETL #ML #PreppingData #MachineLearning #RAG #Engineer #Unstructured #Unstructuredio #RetrievalAugmentedGeneration #multimodal #AIJobs

Unstructured

21,874 views • 2 years ago