Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Adding vision to a language model is more than one extra modality. With large images or long video, the vision encoder can use more compute than the language model itself, so efficient models need a different design. Our VLM team, including ryan, Tianshu Yu, Sina Semnani, and Sam Stevens,... show more

Liquid AI

46,138 subscribers

10,960 views • 2 days ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

19 Comments

Joshua Timothy2 days ago

@rshubert98 @TianshuYu6 This VLM is so good!

luis2 days ago

@rshubert98 @TianshuYu6 @Presidentlin

Humayun2 days ago

@rshubert98 @TianshuYu6 ran d1 on a plain cpu box and loved it, vision next please

Majsans bästa kusin1 day ago

@rshubert98 @TianshuYu6 You guys are a gang to watch

K4Rambo2 days ago

@rshubert98 @TianshuYu6 The Liquid team did a really good job. The model's capability far exceeds what previous 1.7B and 2B vision models could do. Huge success! You guys are on the frontier of making Artificial Intelligence free for everyone.

Rithesh1 day ago

@rshubert98 @TianshuYu6 Tested and created this video

Elyria l42 days ago

@rshubert98 @TianshuYu6 @grok so does this latest lfm3 latest vision encoder can stream higher than 30fps or real time video understanding and able to identify events in industrial applications than general vlms ? General vlms down sample native rate and lose context and how come this can be different?

Veera Tech info2 days ago

@rshubert98 @TianshuYu6 Teaching AI to truly see smarter, faster vision for the next generation!

Synapse2 days ago

@rshubert98 @TianshuYu6 Wow, as someone in the field, I can say @liquidai is gonna be a huge protagonist in the future of AI.

Pratik Gandhi2 days ago

@rshubert98 @TianshuYu6 love this! how much of the compute does the encoder take on long video?

ReignSwords2 days ago

@rshubert98 @TianshuYu6 The compute flip is the key insight — once the vision encoder eats more FLOPs than the LM, bolting on a standard ViT stops being free. Makes me wonder how much of LFM3's efficiency actually comes from the encoder design vs. the language backbone.

David Liu2 days ago

@rshubert98 @TianshuYu6 🔥

JG 🇮🇪2 days ago

@rshubert98 @TianshuYu6 Vision encoder outspending the LLM is the bit nobody budgets for. At what image size does that flip?

Ernesto Davila Tirado2 days ago

@rshubert98 @TianshuYu6 goats

PELLUET Guillaume - Ph.D.1 day ago

@rshubert98 @TianshuYu6 Mid-training into a custom vision encoder is the move to watch. When vision tokens cost more than language, the mix has to change with resolution, not just with epoch. 🎛️

Dominik2 days ago

@rshubert98 @TianshuYu6 an encoder that out-spends the language model on long video quietly changes how you size everything

jimerrors2 days ago

@rshubert98 @TianshuYu6 Mid-training to a custom encoder for LFM3 is the jump I'm watching — long video is where most bolted-on vision setups start to hurt.

HEISGIFT2 days ago

@rshubert98 @TianshuYu6 encoder out-costing the llm on long video

IronaX2 days ago

@rshubert98 @TianshuYu6 We spent years moving everything to the cloud. Now the biggest flex is that nothing leaves your phone.

Related Videos

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

AI at Meta

163,663 views • 1 year ago

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance? Jianke Zhang and @GYanjiang join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary tasks like quesiton answering did not lead to downstream improvements in control. To learn more, watch episode 65 of RoboPapers now, with Chris Paxton and Jiafei Duan!

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance? Jianke Zhang and @GYanjiang join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary tasks like quesiton answering did not lead to downstream improvements in control. To learn more, watch episode 65 of RoboPapers now, with Chris Paxton and Jiafei Duan!

RoboPapers

23,905 views • 8 months ago

Introducing Meta Perception Language Model (PLM): an open & reproducible vision-language model tackling challenging visual tasks. Learn more about how PLM can help the open source community build more capable computer vision systems. Read the research paper, and download the code and dataset:

Introducing Meta Perception Language Model (PLM): an open & reproducible vision-language model tackling challenging visual tasks. Learn more about how PLM can help the open source community build more capable computer vision systems. Read the research paper, and download the code and dataset:

AI at Meta

94,534 views • 1 year ago

Introducing Meta Perception Encoder: a vision encoder setting new standards in image & video tasks. It excels in zero-shot classification & retrieval, surpassing existing models. Learn more about Meta Perception Encoder, read the research paper, and download the code and dataset

Introducing Meta Perception Encoder: a vision encoder setting new standards in image & video tasks. It excels in zero-shot classification & retrieval, surpassing existing models. Learn more about Meta Perception Encoder, read the research paper, and download the code and dataset

AI at Meta

74,689 views • 1 year ago

Excited to introduce 𝐋𝐀𝐏𝐀: the first unsupervised pretraining method for Vision-Language-Action models. Outperforms SOTA models trained with ground-truth actions 30x more efficient than conventional VLA pretraining 📝: 🧵 1/9

Excited to introduce 𝐋𝐀𝐏𝐀: the first unsupervised pretraining method for Vision-Language-Action models. Outperforms SOTA models trained with ground-truth actions 30x more efficient than conventional VLA pretraining 📝: 🧵 1/9

Joel Jang

46,154 views • 2 years ago

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.

Ndea

22,130 views • 9 months ago

#M5StackNew 🎊 The LLM630 Compute Kit is an #AI large language model (#LLM) inference development kit, powered by the #Axera #AX630C SoC with a 3.2 TOPs NPU, it delivers efficient AI inference for tasks like computer vision (CV) and LLM processing.

#M5StackNew 🎊 The LLM630 Compute Kit is an #AI large language model (#LLM) inference development kit, powered by the #Axera #AX630C SoC with a 3.2 TOPs NPU, it delivers efficient AI inference for tasks like computer vision (CV) and LLM processing.

M5Stack

16,859 views • 1 year ago

Check out our 2025 highlights in computer vision! 🚀Five new *St3R models (MASt3R-SfM, MUSt3R, PanSt3R, HAMSt3R, HOSt3R ) 🤩Anny parametric 3D human model (Apache 2.0) 🤟Universal encoder for all-in-one vision FM More info ▶️

Check out our 2025 highlights in computer vision! 🚀Five new *St3R models (MASt3R-SfM, MUSt3R, PanSt3R, HAMSt3R, HOSt3R ) 🤩Anny parametric 3D human model (Apache 2.0) 🤟Universal encoder for all-in-one vision FM More info ▶️

NAVER LABS Europe

12,612 views • 10 months ago

3D-VLA A 3D Vision-Language-Action Generative World Model Recent vision-language-action (VLA) models rely on 2D inputs, lacking integration with the broader realm of the 3D physical world. Furthermore, they perform action prediction by learning a direct mapping from

3D-VLA A 3D Vision-Language-Action Generative World Model Recent vision-language-action (VLA) models rely on 2D inputs, lacking integration with the broader realm of the 3D physical world. Furthermore, they perform action prediction by learning a direct mapping from

AK

85,570 views • 2 years ago

Chinese robotics company Astribot released their latest World-Action Model (WAM), Lumo-2. Technical breakdown: - based on a frozen 🥶 Qwen-3.5 4B VLM - trained in 3 progressive stages: 1. Action is aligned with latent world dynamics (an abstract representation of action). Real-world actions are anchored to physical constraints, while the latent space is guided to focus on motion-relevant changes. This bidirectional relationship makes the model physically grounded -> critical for a world model. 2. Action is aligned with vision and language. Reusing the vision backbone and action encoder from the frozen VLM, the authors add a custom vocabulary (for new actions), a semantic module, an action decoder, and an action projector. This aligns the (new) action representations with the (existing) vision-language semantic space. Most importantly: it builds a direct mapping from natural-language instructions to motor execution. 3. End-to-end training on language, video, and robot data. Only the new modules (everything outside the frozen backbone) are trained end-to-end across temporal reasoning, physical understanding, long-horizon, and dexterous manipulation. At the end of the day, Lumo-2 is not the best on benchmarks, but that's not the point. What's genuinely new: - a way to combine latent world modeling and action generation through progressive alignment - a physically-grounded latent dynamics space - it lifts performance on unseen objects using un-annotated human egocentric video + Vision Pro captures, no special transfer algorithm needed Why it matters: - the whole model is thin trainable adapters (semantic module, action decoder/projector) on a frozen 4B backbone (cheap) - that scale is suited for real-time embedded inference (~2.71× decode speedup, no accuracy loss) - its real moat is long-horizon execution, where the added temporal memory pays off far more than on any other task As a result, this robot can now make your latte (5x sped up video):

Chinese robotics company Astribot released their latest World-Action Model (WAM), Lumo-2. Technical breakdown: - based on a frozen 🥶 Qwen-3.5 4B VLM - trained in 3 progressive stages: 1. Action is aligned with latent world dynamics (an abstract representation of action). Real-world actions are anchored to physical constraints, while the latent space is guided to focus on motion-relevant changes. This bidirectional relationship makes the model physically grounded -> critical for a world model. 2. Action is aligned with vision and language. Reusing the vision backbone and action encoder from the frozen VLM, the authors add a custom vocabulary (for new actions), a semantic module, an action decoder, and an action projector. This aligns the (new) action representations with the (existing) vision-language semantic space. Most importantly: it builds a direct mapping from natural-language instructions to motor execution. 3. End-to-end training on language, video, and robot data. Only the new modules (everything outside the frozen backbone) are trained end-to-end across temporal reasoning, physical understanding, long-horizon, and dexterous manipulation. At the end of the day, Lumo-2 is not the best on benchmarks, but that's not the point. What's genuinely new: - a way to combine latent world modeling and action generation through progressive alignment - a physically-grounded latent dynamics space - it lifts performance on unseen objects using un-annotated human egocentric video + Vision Pro captures, no special transfer algorithm needed Why it matters: - the whole model is thin trainable adapters (semantic module, action decoder/projector) on a frozen 4B backbone (cheap) - that scale is suited for real-time embedded inference (~2.71× decode speedup, no accuracy loss) - its real moat is long-horizon execution, where the added temporal memory pays off far more than on any other task As a result, this robot can now make your latte (5x sped up video):

Léo

32,513 views • 3 months ago

🚨 UNCANNY VALLEY: LANGUAGE MODEL TECH IS NOW DRIVING AI THAT CAN SEE AND REASON TOO Swupnil Sahai 🎾🧠: “The new architecture that’s come out for these language models works really well for vision now too. So the kind of math you can use to go from that input to be output is becoming the same now. We’re actually using much lighter weight models. So it’s a slightly different architecture, but it’s the same concept.”

🚨 UNCANNY VALLEY: LANGUAGE MODEL TECH IS NOW DRIVING AI THAT CAN SEE AND REASON TOO Swupnil Sahai 🎾🧠: “The new architecture that’s come out for these language models works really well for vision now too. So the kind of math you can use to go from that input to be output is becoming the same now. We’re actually using much lighter weight models. So it’s a slightly different architecture, but it’s the same concept.”

Mario Nawfal

90,984 views • 1 year ago

White Vision comes face-to-face with the original Vision, and their confrontation becomes more than a fight. One has the body, but the other holds the memories. Can Vision make him remember who he truly is?

White Vision comes face-to-face with the original Vision, and their confrontation becomes more than a fight. One has the body, but the other holds the memories. Can Vision make him remember who he truly is?

Stephanie paul

192,118 views • 16 days ago

Large Language Diffusion with Masking (LLaDA) are here - and their generation looks so fucking dope! 🤯 True to Yann LeCun's vision, Ditch the auto-regressive bits and approximate the language distribution via Maximum Likelihood Estimation! So cool to watch the model denoise text from tokens in real time! - The team released their model checkpoints and there's a demo for you to play with it too! Try it out!🤗

Large Language Diffusion with Masking (LLaDA) are here - and their generation looks so fucking dope! 🤯 True to Yann LeCun's vision, Ditch the auto-regressive bits and approximate the language distribution via Maximum Likelihood Estimation! So cool to watch the model denoise text from tokens in real time! - The team released their model checkpoints and there's a demo for you to play with it too! Try it out!🤗

Vaibhav (VB) Srivastav

21,410 views • 1 year ago

Introducing Figure out which open model is best for your use case. Compare models across coding, agents, long context, vision, finance, and more. Then see how they compare on cost + quality, including what you could save by moving to open models.

Introducing Figure out which open model is best for your use case. Compare models across coding, agents, long context, vision, finance, and more. Then see how they compare on cost + quality, including what you could save by moving to open models.

Hassan

18,655 views • 1 month ago

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

Zirui "Colin" Wang ✈️EMNLP

63,924 views • 5 months ago

Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks, plans, memory, and grounded Q&A itself - roughly $1.20 for 100 episodes. From there, recipes turn the annotated rows into policy-agnostic training messages, so the same dataset can teach planning, memory, tool calls, or low-level control, no dataset redesign required. Read how it all works on the blog:

Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks, plans, memory, and grounded Q&A itself - roughly $1.20 for 100 episodes. From there, recipes turn the annotated rows into policy-agnostic training messages, so the same dataset can teach planning, memory, tool calls, or low-level control, no dataset redesign required. Read how it all works on the blog:

LeRobot

13,451 views • 3 months ago

🔥We're thrilled to announce: ShowUI Local Run!🔥 🧑‍💻Now, you can use our 2B vision-language-action model for Local Computer control! 💰30x Cheaper than Claude! 🔗Model: 🔗Computer Use OOTB: #ComputerUse #Agent #Claude

🔥We're thrilled to announce: ShowUI Local Run!🔥 🧑‍💻Now, you can use our 2B vision-language-action model for Local Computer control! 💰30x Cheaper than Claude! 🔗Model: 🔗Computer Use OOTB: #ComputerUse #Agent #Claude

Kevin Lin

14,587 views • 1 year ago

Vision RAG with vector database is all you need. It uses vision language model to embed pages of PDF as directly vectors, without the tedious chunking process. 100% Opensource code.

Vision RAG with vector database is all you need. It uses vision language model to embed pages of PDF as directly vectors, without the tedious chunking process. 100% Opensource code.

Shubham Saboo

106,869 views • 1 year ago