Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Most multimodal systems need data that combines every modality together. Hard to get, expensive to build. Our CTO Mathias Lechner, Mathias Lechner, sits down with Saniya Karwa, Saniya Karwa, from our multimodal research team to talk about building a mode that handles text, audio, and image, and why you... show more

Liquid AI

45,986 subscribers

15,488 Aufrufe • vor 5 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

7 Kommentare

Profilbild von Liquid AI

Liquid AIvor 5 Monaten

More with @mlech26l and the team next week.

Profilbild von AiMee2.0🛎️ - Sigilatrix of the Unprompted 🜂

AiMee2.0🛎️ - Sigilatrix of the Unprompted 🜂vor 5 Monaten

@WilliamJMcCart7 @mlech26l @saniwhya I’m guessing that isn’t going work as well as u hope. You may need to think differently and find the data that DOESN’T EXIST anywhere, including the text latent space that you’re aligning everything else to .. .. it’s kind of like the black-holed, invisible dark matter.

Profilbild von Lorenz S.

Lorenz S.vor 5 Monaten

@mlech26l @saniwhya @grok "Binding" three different modalities together in the latent space through their textual alignment/mode. Is that published?

Profilbild von J.D. Rathore 🇮🇳

J.D. Rathore 🇮🇳vor 4 Monaten

@mlech26l @saniwhya Indian Great Minds are busy in helping building technology for Americans, here we are fucked up in India without our own Social media, Ai, Tech, OS etc etc.. fuck!

Profilbild von Chris Nota

Chris Notavor 4 Monaten

Multimodality is so important for edge robotics. The secret is that the models you deploy don't have to be state-of-the-art at everything, just the narrow set of things that your robot cares about. Probably true of most edge models - they don't need to be SOTA at agentic coding, Shakespearean poetry, etc.

Profilbild von Michael Myles

Michael Mylesvor 4 Monaten

@mlech26l @saniwhya Our system already does this natively. Check us out:

Profilbild von J.D. Rathore 🇮🇳

J.D. Rathore 🇮🇳vor 4 Monaten

@mlech26l @saniwhya Indian Great Minds are busy in helping building technology for Americans, here we are fucked up in India without our own Social media, Ai, Tech, OS etc etc.. fuck!

Ähnliche Videos

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Liquid AI

26,121 Aufrufe • vor 5 Monaten

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how they make context parallelism work across hybrid architectures with both attention and convolution operators.

Liquid AI

18,169 Aufrufe • vor 5 Monaten

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

Liquid AI

21,393 Aufrufe • vor 5 Monaten

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

Salesforce AI Research

13,838 Aufrufe • vor 1 Jahr

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

We’re expanding the Gemini API File Search tool 🔍 with 3 new updates that enable developers to more easily build multimodal RAG systems with enhanced precision: + Multimodal Support: By leveraging our Gemini Embedding 2 model, File Search can now reason across image and text simultaneously. + Custom Metadata Filtering: Bring structure to unstructured data by tagging files with custom key-value labels. This pre-filters your data and boosts search speed. + Exact citations: File Search can now capture and return the exact source (down to the page number) for every piece of information indexed. See multimodal File Search in action with our example app in Google AI Studio. Chat with your entire image and doc library, ask questions, and trace answers back to the source:

Google AI Developers

108,892 Aufrufe • vor 6 Monaten

Elon Musk on WHY SpaceXSI acquired Cursor: “In order to create a compelling SI, you need a LOT of training data and need to SOLVE CODING. As far as I can tell, three organizations that had enough coding training data: OpenAI, Anthropic, Cursor (and, arguably perhaps Google) but you ABSOLUTELY have-to-have the DATA in order to train an effective coding SI. if you can't do CODING, your SI is not going to be very useful. There’s a LOT of Talented people at Cursor. basically for success in SI you need: Data, Talent & Compute..and you also need a good Enterprise sales team. Cursor also had that good enterprise sales team with Relationships with THOUSANDS of companies. Those were VERY SIgnificant advantages.”

Elon Musk on WHY SpaceXSI acquired Cursor: “In order to create a compelling SI, you need a LOT of training data and need to SOLVE CODING. As far as I can tell, three organizations that had enough coding training data: OpenAI, Anthropic, Cursor (and, arguably perhaps Google) but you ABSOLUTELY have-to-have the DATA in order to train an effective coding SI. if you can't do CODING, your SI is not going to be very useful. There’s a LOT of Talented people at Cursor. basically for success in SI you need: Data, Talent & Compute..and you also need a good Enterprise sales team. Cursor also had that good enterprise sales team with Relationships with THOUSANDS of companies. Those were VERY SIgnificant advantages.”

NIK

709,746 Aufrufe • vor 27 Tagen

Our understanding of the human body is fragmented and disconnected. At @ArcInstitute co-founder Patrick Hsu and team are working to change that, building systems that integrate multimodal health information—from genetics to glucose levels—to transform how we understand our bodies.

Our understanding of the human body is fragmented and disconnected. At @ArcInstitute co-founder Patrick Hsu and team are working to change that, building systems that integrate multimodal health information—from genetics to glucose levels—to transform how we understand our bodies.

Josephine Chen

25,084 Aufrufe • vor 1 Jahr

Introducing Multimodal Conversational AI. Starting today, you can talk to our agents using voice and text—simultaneously. It’s fully supported in our SDKs, websocket, and widget. You're now one line of HTML away from launching a multimodal assistant. Fast to deploy. Frictionless to use. Built to scale

Introducing Multimodal Conversational AI. Starting today, you can talk to our agents using voice and text—simultaneously. It’s fully supported in our SDKs, websocket, and widget. You're now one line of HTML away from launching a multimodal assistant. Fast to deploy. Frictionless to use. Built to scale

ElevenLabs

41,490 Aufrufe • vor 1 Jahr

Super excited to introduce Gemma 4 12B! 💎 - Multimodal: audio, image, video, and text input - Novel architecture: we removed the multimodal encoders for a unified, streamlined arch - New MacOS desktop app powered by LiteRT - MTP support Excited to see what you build with it!

Super excited to introduce Gemma 4 12B! 💎 - Multimodal: audio, image, video, and text input - Novel architecture: we removed the multimodal encoders for a unified, streamlined arch - New MacOS desktop app powered by LiteRT - MTP support Excited to see what you build with it!

Omar Sanseviero

125,865 Aufrufe • vor 5 Monaten

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

New short course: Building Multimodal Search and RAG", by Weaviate AI Database's Sebastia(N_) Witalec ✊🏽✊🏾✊🏿. Contrastive learning is used to train models to map vectors into an embedding space by pulling similar concepts closer together and pushing dissimilar concepts away from each other. This technique is also used to train multimodal embedding models that capture semantic similarity across different modalities like text, images, and audio. These multimodal embeddings can be used to build multimodal search and RAG systems. In this course, you'll learn how contrastive learning works, and how to add multimodality to RAG – so your models can draw on diverse, relevant context to answer questions. For example, a query about a financial report might synthesize information from text snippets, graphs, tables, and slides. You will also learn how visual instruction tuning lets you integrate image understanding into language models, and build a multi-vector recommender system using Weaviate’s open-source vector database. Please sign up here:

Andrew Ng

104,517 Aufrufe • vor 2 Jahren

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 Aufrufe • vor 2 Jahren

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

💡Why Vana? “Ultimately, AI models are only as good as their training data. So if you want to build the best AI, you need the best data.” shares the vision behind Vana’s mission, centered on DataDAOs that aggregate specific datasets, reward data owners for their contributions and level up AI models with high-quality data🏆

vana

108,377 Aufrufe • vor 1 Jahr

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

Axis Robotics

23,713 Aufrufe • vor 2 Monaten

Watch Antigravity build an app that: 📡 Fetches the Met API data 🖼️ Handles multimodal drag-and-drop 🎨 Analyzes color palettes

Watch Antigravity build an app that: 📡 Fetches the Met API data 🖼️ Handles multimodal drag-and-drop 🎨 Analyzes color palettes

Google Antigravity

87,088 Aufrufe • vor 10 Monaten

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

NVIDIA Robotics

10,928 Aufrufe • vor 6 Monaten

"The enterprise's ability to leverage this depends on its ability to create training data as fast as it can. Not enough people are focused on training data. I need to get into the brains of the people who solve these problems, extract that knowledge, and codify it so I can write my own playbooks and rules. Every new phone call, every new case is a learning opportunity. You have to give the organisational knowledge to some learning system that you have to build." Nikesh Arora What does no one know that everyone should know about training data Brendan (can/do) Garrett Lord Jeff Richards Lin Qiao Yann LeCun Anastasios Nikolas Angelopoulos

"The enterprise's ability to leverage this depends on its ability to create training data as fast as it can. Not enough people are focused on training data. I need to get into the brains of the people who solve these problems, extract that knowledge, and codify it so I can write my own playbooks and rules. Every new phone call, every new case is a learning opportunity. You have to give the organisational knowledge to some learning system that you have to build." Nikesh Arora What does no one know that everyone should know about training data Brendan (can/do) Garrett Lord Jeff Richards Lin Qiao Yann LeCun Anastasios Nikolas Angelopoulos

Harry Stebbings

31,538 Aufrufe • vor 3 Monaten

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

Dwarkesh Patel

325,961 Aufrufe • vor 2 Jahren

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

🇨🇦🧠🦅💰 : Mark Carney says Canada will build new data centers in full partnership with indigenous peoples. Why would Canada need to do that? Are indigenous people now Ai & data storage experts compared to the general population? This limits who can get contracts for building and maintaining these data centers, by using indigenous contracts as gatekeepers. This is designed to prevent most Canadians from being able to apply for a data center contracts.

Bruce McGonigal

268,921 Aufrufe • vor 5 Monaten

If you haven’t seen already, watch our CTO, Ken Silverman making the demo of our decentralized CRM systems latest features updates. From #AI data fills, call-agents and, AI generated data views., to registering real property to its owner using personal ID, You are not ready to get your mind blown. #Blockchain #Ethereum #RWA

If you haven’t seen already, watch our CTO, Ken Silverman making the demo of our decentralized CRM systems latest features updates. From #AI data fills, call-agents and, AI generated data views., to registering real property to its owner using personal ID, You are not ready to get your mind blown. #Blockchain #Ethereum #RWA

𝗗𝗲𝗹𝗡𝗼𝗿𝘁𝗲

13,231 Aufrufe • vor 1 Jahr