Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Most multimodal systems need data that combines every modality together. Hard to get, expensive to build. Our CTO Mathias Lechner, Mathias Lechner, sits down with Saniya Karwa, Saniya Karwa, from our multimodal research team to talk about building a mode that handles text, audio, and image, and why you... show more

Liquid AI

35,753 subscribers

15,382 views • 2 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Liquid AI

26,121 views • 3 months ago

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Liquid AI

17,908 views • 2 months ago

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

Salesforce AI Research

13,838 views • 1 year ago

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

Google AI Developers

108,801 views • 3 months ago

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 views • 2 years ago

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

vana

108,377 views • 1 year ago

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

Andrew Ng

104,371 views • 2 years ago

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

Axis Robotics

22,634 views • 18 days ago

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

NVIDIA Robotics

10,840 views • 4 months ago

"The enterprise's ability to leverage this depends on its ability to create training data as fast as it can. Not enough people are focused on training data. I need to get into the brains of the people who solve these problems, extract that knowledge, and codify it so I can write my own playbooks and rules. Every new phone call, every new case is a learning opportunity. You have to give the organisational knowledge to some learning system that you have to build." Nikesh Arora What does no one know that everyone should know about training data Brendan (can/do) Garrett Lord Jeff Richards Lin Qiao Yann LeCun Anastasios Nikolas Angelopoulos

"The enterprise's ability to leverage this depends on its ability to create training data as fast as it can. Not enough people are focused on training data. I need to get into the brains of the people who solve these problems, extract that knowledge, and codify it so I can write my own playbooks and rules. Every new phone call, every new case is a learning opportunity. You have to give the organisational knowledge to some learning system that you have to build." Nikesh Arora What does no one know that everyone should know about training data Brendan (can/do) Garrett Lord Jeff Richards Lin Qiao Yann LeCun Anastasios Nikolas Angelopoulos

Harry Stebbings

31,538 views • 22 days ago

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

Dwarkesh Patel

325,893 views • 2 years ago

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

Bruce McGonigal

268,745 views • 2 months ago

Thank you! That is why I need every single South African to vote for the EFF. As poor black people we need to trust Malema with our votes. And I'm not talking to you sellouts, y'all can continue selling out. We don't care Msunu

Thank you! That is why I need every single South African to vote for the EFF. As poor black people we need to trust Malema with our votes. And I'm not talking to you sellouts, y'all can continue selling out. We don't care Msunu

Msunu ka Floyd Shivambu

10,966 views • 3 months ago

New generative UI experiences in the Google Gemini and AI Mode in Search use Gemini 3’s multimodal understanding capabilities to make entire user interfaces for you from scratch — dynamically delivering what you need in a visual and interactive way. Google Research software engineer Yaniv Leviathan shares more about the process of building generative UI (hint: so much fun that the team couldn't stop playing with it) and how he's been using it so far.

New generative UI experiences in the Google Gemini and AI Mode in Search use Gemini 3’s multimodal understanding capabilities to make entire user interfaces for you from scratch — dynamically delivering what you need in a visual and interactive way. Google Research software engineer Yaniv Leviathan shares more about the process of building generative UI (hint: so much fun that the team couldn't stop playing with it) and how he's been using it so far.

Google

138,435 views • 9 months ago

🚀 Introducing Genspark AgentBase (Preview). Turn your data into custom databases, dashboards, and internal systems. Stop buying 30+ SaaS tools. Build your own with Genspark AgentBase. - Compatible with your current systems: Salesforce, HubSpot, and your other existing databases. - Pull data from your daily work: inbox, files, apps, and meeting notes. - Customize your system with one prompt. Tell AgentBase what you need, and it builds dashboards and workflows that match how you actually work. - Build any system you need in minutes: CRM, hiring system, project tracker, and more. Genspark AgentBase makes every SaaS work for you. Start building at our launch price:

🚀 Introducing Genspark AgentBase (Preview). Turn your data into custom databases, dashboards, and internal systems. Stop buying 30+ SaaS tools. Build your own with Genspark AgentBase. - Compatible with your current systems: Salesforce, HubSpot, and your other existing databases. - Pull data from your daily work: inbox, files, apps, and meeting notes. - Customize your system with one prompt. Tell AgentBase what you need, and it builds dashboards and workflows that match how you actually work. - Build any system you need in minutes: CRM, hiring system, project tracker, and more. Genspark AgentBase makes every SaaS work for you. Start building at our launch price:

Genspark

250,041 views • 2 months ago

Yann LeCun -- Meta's chief AI scientist and a Turing Award winner -- explains why scaling up LLMs will never reach human-level AI, and names the four things today's systems still can't do: "If you think that we're going to get to human-level AI by just training on more data and scaling up LLMs, you're making a mistake." "If you're an investor and you invest in a company that told you we're going to get to human-level AI and PhD level by just training on more data and with a few tricks... that was probably not a good idea." "There are ideas about how to go forward and have systems that are capable of doing what every intelligent animal and human are capable of doing, and that current AI systems are not capable of doing." "Understanding the physical world, having persistent memory, and being able to reason and plan. Those are the four characteristics that need to be there." "That requires systems that can acquire common sense, that can learn from natural sensors like video, as opposed to just text." The entire market is priced on a straight line from bigger models to AGI. One of the people who invented deep learning is telling you the line doesn't reach -- and that the real bottleneck isn't compute, it's world models. If he's right, the winner isn't whoever stacks the most GPUs. It's whoever teaches a machine to learn from video the way a child does.

Yann LeCun -- Meta's chief AI scientist and a Turing Award winner -- explains why scaling up LLMs will never reach human-level AI, and names the four things today's systems still can't do: "If you think that we're going to get to human-level AI by just training on more data and scaling up LLMs, you're making a mistake." "If you're an investor and you invest in a company that told you we're going to get to human-level AI and PhD level by just training on more data and with a few tricks... that was probably not a good idea." "There are ideas about how to go forward and have systems that are capable of doing what every intelligent animal and human are capable of doing, and that current AI systems are not capable of doing." "Understanding the physical world, having persistent memory, and being able to reason and plan. Those are the four characteristics that need to be there." "That requires systems that can acquire common sense, that can learn from natural sensors like video, as opposed to just text." The entire market is priced on a straight line from bigger models to AGI. One of the people who invented deep learning is telling you the line doesn't reach -- and that the real bottleneck isn't compute, it's world models. If he's right, the winner isn't whoever stacks the most GPUs. It's whoever teaches a machine to learn from video the way a child does.

Karl Mehta

34,797 views • 1 month ago

⚡️We are excited to announce that our new no-code Enterprise Platform is NOW available in private beta! As RAG apps advance from prototype to production we’ve been overwhelmed by requests for an enterprise grade solution to provide these applications with the data they need. Designed to make it easy to get your data #RAGready, our Platform can preprocess more than 25 file types and soon will be fully #multimodal, also able to ingest audio, video and image files. We ship with a baseline suite of source connectors, including Amazon Web Services S3, Microsoft Azure Blob Storage, OneDrive, SFTP, Databricks Delta Table, Google Drive, Salesforce, Elastic, OpenSearch, and Google Cloud storage with many more fast following. Platform transforms your documents into a standardized JSON schema, broken down into semantically coherent elements allowing you to reconstruct your document in the manner most useful to you. Want only the narrative text but not the headers and footers? This is entirely configurable through the UI. Additionally, we generate more than 30 types of metadata for each element to make it easy to curate the data being written downstream and to support metadata filtering during retrieval. Smart chunking and the ability to choose from a range of embedding models are in from launch, delivering a turnkey solution for chunk and embedding experimentation. As for destination connectors, we've got that covered too, with Amazon Web Services S3, Pinecone, Chroma , Weaviate AI Database, Google Cloud storage, MongoDB, Microsoft Azure cognitive search, PostgreSQL, Elastic, OpenSearch, and Databricks Delta Table. And of course, all of this can be scheduled to keep your data continuously hydrated. The private-beta is live today! Sign-up to get access and come build the future of LLM data foundations with us: 🚀 #ETLforLLMs #AI #DataPreprocessing #DataScience #DataTransformation #LLMs #ETL #ML #PreppingData #MachineLearning #RAG #Engineer #Unstructured #Unstructuredio #RetrievalAugmentedGeneration #multimodal #AIJobs

⚡️We are excited to announce that our new no-code Enterprise Platform is NOW available in private beta! As RAG apps advance from prototype to production we’ve been overwhelmed by requests for an enterprise grade solution to provide these applications with the data they need. Designed to make it easy to get your data #RAGready, our Platform can preprocess more than 25 file types and soon will be fully #multimodal, also able to ingest audio, video and image files. We ship with a baseline suite of source connectors, including Amazon Web Services S3, Microsoft Azure Blob Storage, OneDrive, SFTP, Databricks Delta Table, Google Drive, Salesforce, Elastic, OpenSearch, and Google Cloud storage with many more fast following. Platform transforms your documents into a standardized JSON schema, broken down into semantically coherent elements allowing you to reconstruct your document in the manner most useful to you. Want only the narrative text but not the headers and footers? This is entirely configurable through the UI. Additionally, we generate more than 30 types of metadata for each element to make it easy to curate the data being written downstream and to support metadata filtering during retrieval. Smart chunking and the ability to choose from a range of embedding models are in from launch, delivering a turnkey solution for chunk and embedding experimentation. As for destination connectors, we've got that covered too, with Amazon Web Services S3, Pinecone, Chroma , Weaviate AI Database, Google Cloud storage, MongoDB, Microsoft Azure cognitive search, PostgreSQL, Elastic, OpenSearch, and Databricks Delta Table. And of course, all of this can be scheduled to keep your data continuously hydrated. The private-beta is live today! Sign-up to get access and come build the future of LLM data foundations with us: 🚀 #ETLforLLMs #AI #DataPreprocessing #DataScience #DataTransformation #LLMs #ETL #ML #PreppingData #MachineLearning #RAG #Engineer #Unstructured #Unstructuredio #RetrievalAugmentedGeneration #multimodal #AIJobs

Unstructured

21,874 views • 2 years ago

Data centers in space are a "no brainer." SpaceX's 1st employee Tom Mueller (Tom Mueller) on Elon Musk's plan to move compute into space: "It makes the most sense of anything to move to space. All you need as an input is power, and all you have as an output is data." "You move it to space, you have all the power you would ever need, and you transmit that data back down on a terawatt laser beam—it's solved. It's just so simple."

Data centers in space are a "no brainer." SpaceX's 1st employee Tom Mueller (Tom Mueller) on Elon Musk's plan to move compute into space: "It makes the most sense of anything to move to space. All you need as an input is power, and all you have as an output is data." "You move it to space, you have all the power you would ever need, and you transmit that data back down on a terawatt laser beam—it's solved. It's just so simple."

Molly O’Shea

357,597 views • 2 months ago

We're coming out of stealth to announce our cyber defense research lab. We are exploring data and post-training techniques to build superhuman cyber defenders. Our mission is to make sure the West always wins. The last 3 months we've built an automated data pipeline to create training data from 80k CVEs (aka public vulnerabilities). Our next topic? Post training a model that's better at fixing all the vulnerabilities in your codebase. Like really fixing them. Not saying it's secure when there are still ways to exploit them. Here are the questions that keep us awake at night: How do you train a model to defend without improving its capabilities to attack? What's the right reward? How to measure the defense capabilities? How do you create synth training data that reproduces real systems? What kind of access do you give an ai cyber defender? How far can you trust it? If you know insanely good cyber experts (red team, blue team, CTF aficionados) or ML engineers (synth data generation and post-training models), send them my way. We need to make models far better at defending.

We're coming out of stealth to announce our cyber defense research lab. We are exploring data and post-training techniques to build superhuman cyber defenders. Our mission is to make sure the West always wins. The last 3 months we've built an automated data pipeline to create training data from 80k CVEs (aka public vulnerabilities). Our next topic? Post training a model that's better at fixing all the vulnerabilities in your codebase. Like really fixing them. Not saying it's secure when there are still ways to exploit them. Here are the questions that keep us awake at night: How do you train a model to defend without improving its capabilities to attack? What's the right reward? How to measure the defense capabilities? How do you create synth training data that reproduces real systems? What kind of access do you give an ai cyber defender? How far can you trust it? If you know insanely good cyber experts (red team, blue team, CTF aficionados) or ML engineers (synth data generation and post-training models), send them my way. We need to make models far better at defending.

Paco Villetard

120,215 views • 4 months ago