Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Most multimodal systems need data that combines every modality together. Hard to get, expensive to build. Our CTO Mathias Lechner, Mathias Lechner, sits down with Saniya Karwa, Saniya Karwa, from our multimodal research team to talk about building a mode that handles text, audio, and image, and why you... show more

Liquid AI

31,398 subscribers

15,110 views • 1 month ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Liquid AI

25,877 views • 1 month ago

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Liquid AI

17,864 views • 1 month ago

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

Liquid AI

21,393 views • 1 month ago

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

Salesforce AI Research

13,838 views • 1 year ago

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

Google AI Developers

108,622 views • 2 months ago

Our understanding of the human body is fragmented and disconnected. At @ArcInstitute co-founder Patrick Hsu and team are working to change that, building systems that integrate multimodal health information—from genetics to glucose levels—to transform how we understand our bodies.

Our understanding of the human body is fragmented and disconnected. At @ArcInstitute co-founder Patrick Hsu and team are working to change that, building systems that integrate multimodal health information—from genetics to glucose levels—to transform how we understand our bodies.

Josephine Chen

25,084 views • 1 year ago

Introducing Multimodal Conversational AI. Starting today, you can talk to our agents using voice and text—simultaneously. It’s fully supported in our SDKs, websocket, and widget. You're now one line of HTML away from launching a multimodal assistant. Fast to deploy. Frictionless to use. Built to scale

Introducing Multimodal Conversational AI. Starting today, you can talk to our agents using voice and text—simultaneously. It’s fully supported in our SDKs, websocket, and widget. You're now one line of HTML away from launching a multimodal assistant. Fast to deploy. Frictionless to use. Built to scale

ElevenLabs

41,490 views • 1 year ago

Super excited to introduce Gemma 4 12B! 💎 - Multimodal: audio, image, video, and text input - Novel architecture: we removed the multimodal encoders for a unified, streamlined arch - New MacOS desktop app powered by LiteRT - MTP support Excited to see what you build with it!

Super excited to introduce Gemma 4 12B! 💎 - Multimodal: audio, image, video, and text input - Novel architecture: we removed the multimodal encoders for a unified, streamlined arch - New MacOS desktop app powered by LiteRT - MTP support Excited to see what you build with it!

Omar Sanseviero 🇫🇷 @RAISE Paris

125,341 views • 1 month ago

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 views • 1 year ago

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

Andrew Ng

104,371 views • 2 years ago

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

vana

108,354 views • 1 year ago

Watch Antigravity build an app that: 📡 Fetches the Met API data 🖼️ Handles multimodal drag-and-drop 🎨 Analyzes color palettes

Watch Antigravity build an app that: 📡 Fetches the Met API data 🖼️ Handles multimodal drag-and-drop 🎨 Analyzes color palettes

Google Antigravity

86,755 views • 6 months ago

DoorDash and UberEats need big teams $$$ to clean, merge, and enrich local business data. We act as that team—handling APIs, data cleanup, and enrichment so you don’t have to. Get deep local business insights in seconds. ->

DoorDash and UberEats need big teams $$$ to clean, merge, and enrich local business data. We act as that team—handling APIs, data cleanup, and enrichment so you don’t have to. Get deep local business insights in seconds. ->

Kathryn Wu

11,711 views • 1 year ago

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

NVIDIA Robotics

10,840 views • 2 months ago

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

Dwarkesh Patel

325,857 views • 2 years ago

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

Bruce McGonigal

268,438 views • 1 month ago

We need to get familiar with the ordinary levers of power and areas of resistance within our every day life. We need to learn to recognize and seize opportunities that allow us to act as transgressors against immoral systems. Brightblackhoney

We need to get familiar with the ordinary levers of power and areas of resistance within our every day life. We need to learn to recognize and seize opportunities that allow us to act as transgressors against immoral systems. Brightblackhoney

Jean Jacques Dessalines 🇭🇹🇵🇸🇳🇪🇲🇱🇧🇫🇨🇺

18,463 views • 5 months ago

If you haven’t seen already, watch our CTO, Ken Silverman making the demo of our decentralized CRM systems latest features updates. From #AI data fills, call-agents and, AI generated data views., to registering real property to its owner using personal ID, You are not ready to get your mind blown. #Blockchain #Ethereum #RWA

If you haven’t seen already, watch our CTO, Ken Silverman making the demo of our decentralized CRM systems latest features updates. From #AI data fills, call-agents and, AI generated data views., to registering real property to its owner using personal ID, You are not ready to get your mind blown. #Blockchain #Ethereum #RWA

𝗗𝗲𝗹𝗡𝗼𝗿𝘁𝗲

13,221 views • 1 year ago

“Why do they need to be humanoid robots that freaks me out” Ok? How’s about a Multimodal Hexapod Robot ?

“Why do they need to be humanoid robots that freaks me out” Ok? How’s about a Multimodal Hexapod Robot ?

Brian Roemmele

29,078 views • 6 months ago

can AI write engaging news that people can trust? introducing ✨Data2Story: a data journalist agent. give it raw data, it generate a verifiable, multimodal article. 🔍verifiable: every claim is evidence-grounded, traces back to data, code, or a cited source. 🔮multimodal: the article is a generative UI — images, videos, audio, interactive charts. not just readable, but trustworthy and playable. 🧵1/N

can AI write engaging news that people can trust? introducing ✨Data2Story: a data journalist agent. give it raw data, it generate a verifiable, multimodal article. 🔍verifiable: every claim is evidence-grounded, traces back to data, code, or a cited source. 🔮multimodal: the article is a generative UI — images, videos, audio, interactive charts. not just readable, but trustworthy and playable. 🧵1/N

Kevin Lin

32,195 views • 26 days ago