正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Most multimodal systems need data that combines every modality together. Hard to get, expensive to build. Our CTO Mathias Lechner, Mathias Lechner, sits down with Saniya Karwa, Saniya Karwa, from our multimodal research team to talk about building a mode that handles text, audio, and image, and why you... show more

Liquid AI

45,986 subscribers

15,488 次观看 • 5 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

7 条评论

Liquid AI 的头像

Liquid AI5 个月前

More with @mlech26l and the team next week.

AiMee2.0🛎️ - Sigilatrix of the Unprompted 🜂 的头像

AiMee2.0🛎️ - Sigilatrix of the Unprompted 🜂5 个月前

@WilliamJMcCart7 @mlech26l @saniwhya I’m guessing that isn’t going work as well as u hope. You may need to think differently and find the data that DOESN’T EXIST anywhere, including the text latent space that you’re aligning everything else to .. .. it’s kind of like the black-holed, invisible dark matter.

Lorenz S. 的头像

Lorenz S.5 个月前

@mlech26l @saniwhya @grok "Binding" three different modalities together in the latent space through their textual alignment/mode. Is that published?

J.D. Rathore 🇮🇳 的头像

J.D. Rathore 🇮🇳4 个月前

@mlech26l @saniwhya Indian Great Minds are busy in helping building technology for Americans, here we are fucked up in India without our own Social media, Ai, Tech, OS etc etc.. fuck!

Chris Nota 的头像

Chris Nota4 个月前

Multimodality is so important for edge robotics. The secret is that the models you deploy don't have to be state-of-the-art at everything, just the narrow set of things that your robot cares about. Probably true of most edge models - they don't need to be SOTA at agentic coding, Shakespearean poetry, etc.

Michael Myles 的头像

Michael Myles4 个月前

@mlech26l @saniwhya Our system already does this natively. Check us out:

J.D. Rathore 🇮🇳 的头像

J.D. Rathore 🇮🇳4 个月前

@mlech26l @saniwhya Indian Great Minds are busy in helping building technology for Americans, here we are fucked up in India without our own Social media, Ai, Tech, OS etc etc.. fuck!

相关视频

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Liquid AI

26,121 次观看 • 5 个月前

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Liquid AI

18,169 次观看 • 5 个月前

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

Liquid AI

21,393 次观看 • 5 个月前

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

Salesforce AI Research

13,838 次观看 • 1 年前

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

Google AI Developers

108,892 次观看 • 6 个月前

Elon Musk on WHY SpaceXSI acquired Cursor: “In order to create a compelling SI, you need a LOT of training data and need to SOLVE CODING. As far as I can tell, three organizations that had enough coding training data: OpenAI, Anthropic, Cursor (and, arguably perhaps Google) but you ABSOLUTELY have-to-have the DATA in order to train an effective coding SI. if you can't do CODING, your SI is not going to be very useful. There’s a LOT of Talented people at Cursor. basically for success in SI you need: Data, Talent & Compute..and you also need a good Enterprise sales team. Cursor also had that good enterprise sales team with Relationships with THOUSANDS of companies. Those were VERY SIgnificant advantages.”

Elon Musk on WHY SpaceXSI acquired Cursor: “In order to create a compelling SI, you need a LOT of training data and need to SOLVE CODING. As far as I can tell, three organizations that had enough coding training data: OpenAI, Anthropic, Cursor (and, arguably perhaps Google) but you ABSOLUTELY have-to-have the DATA in order to train an effective coding SI. if you can't do CODING, your SI is not going to be very useful. There’s a LOT of Talented people at Cursor. basically for success in SI you need: Data, Talent & Compute..and you also need a good Enterprise sales team. Cursor also had that good enterprise sales team with Relationships with THOUSANDS of companies. Those were VERY SIgnificant advantages.”

NIK

709,746 次观看 • 27 天前

Our understanding of the human body is fragmented and disconnected. At @ArcInstitute co-founder Patrick Hsu and team are working to change that, building systems that integrate multimodal health information—from genetics to glucose levels—to transform how we understand our bodies.

Our understanding of the human body is fragmented and disconnected. At @ArcInstitute co-founder Patrick Hsu and team are working to change that, building systems that integrate multimodal health information—from genetics to glucose levels—to transform how we understand our bodies.

Josephine Chen

25,084 次观看 • 1 年前

Introducing Multimodal Conversational AI. Starting today, you can talk to our agents using voice and text—simultaneously. It’s fully supported in our SDKs, websocket, and widget. You're now one line of HTML away from launching a multimodal assistant. Fast to deploy. Frictionless to use. Built to scale

Introducing Multimodal Conversational AI. Starting today, you can talk to our agents using voice and text—simultaneously. It’s fully supported in our SDKs, websocket, and widget. You're now one line of HTML away from launching a multimodal assistant. Fast to deploy. Frictionless to use. Built to scale

ElevenLabs

41,490 次观看 • 1 年前

Super excited to introduce Gemma 4 12B! 💎 - Multimodal: audio, image, video, and text input - Novel architecture: we removed the multimodal encoders for a unified, streamlined arch - New MacOS desktop app powered by LiteRT - MTP support Excited to see what you build with it!

Super excited to introduce Gemma 4 12B! 💎 - Multimodal: audio, image, video, and text input - Novel architecture: we removed the multimodal encoders for a unified, streamlined arch - New MacOS desktop app powered by LiteRT - MTP support Excited to see what you build with it!

Omar Sanseviero

125,865 次观看 • 5 个月前

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

Andrew Ng

104,517 次观看 • 2 年前

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 次观看 • 2 年前

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

vana

108,377 次观看 • 1 年前

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

Axis Robotics

23,713 次观看 • 2 个月前

Watch Antigravity build an app that: 📡 Fetches the Met API data 🖼️ Handles multimodal drag-and-drop 🎨 Analyzes color palettes

Watch Antigravity build an app that: 📡 Fetches the Met API data 🖼️ Handles multimodal drag-and-drop 🎨 Analyzes color palettes

Google Antigravity

87,088 次观看 • 10 个月前

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

NVIDIA Robotics

10,928 次观看 • 6 个月前

"The enterprise's ability to leverage this depends on its ability to create training data as fast as it can. Not enough people are focused on training data. I need to get into the brains of the people who solve these problems, extract that knowledge, and codify it so I can write my own playbooks and rules. Every new phone call, every new case is a learning opportunity. You have to give the organisational knowledge to some learning system that you have to build." Nikesh Arora What does no one know that everyone should know about training data Brendan (can/do) Garrett Lord Jeff Richards Lin Qiao Yann LeCun Anastasios Nikolas Angelopoulos

"The enterprise's ability to leverage this depends on its ability to create training data as fast as it can. Not enough people are focused on training data. I need to get into the brains of the people who solve these problems, extract that knowledge, and codify it so I can write my own playbooks and rules. Every new phone call, every new case is a learning opportunity. You have to give the organisational knowledge to some learning system that you have to build." Nikesh Arora What does no one know that everyone should know about training data Brendan (can/do) Garrett Lord Jeff Richards Lin Qiao Yann LeCun Anastasios Nikolas Angelopoulos

Harry Stebbings

31,538 次观看 • 3 个月前

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

Dwarkesh Patel

325,961 次观看 • 2 年前

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

Bruce McGonigal

268,921 次观看 • 5 个月前

If you haven’t seen already, watch our CTO, Ken Silverman making the demo of our decentralized CRM systems latest features updates. From #AI data fills, call-agents and, AI generated data views., to registering real property to its owner using personal ID, You are not ready to get your mind blown. #Blockchain #Ethereum #RWA

If you haven’t seen already, watch our CTO, Ken Silverman making the demo of our decentralized CRM systems latest features updates. From #AI data fills, call-agents and, AI generated data views., to registering real property to its owner using personal ID, You are not ready to get your mind blown. #Blockchain #Ethereum #RWA

𝗗𝗲𝗹𝗡𝗼𝗿𝘁𝗲

13,231 次观看 • 1 年前