正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

What makes an on-device agentic model actually useful? Post-training. Maxime Labonne, Edoardo Mosca, and Jiahui Wang from Liquid’s post-training team discuss how models learn to use tools, follow instructions, handle longer contexts, and recover when tasks get complex.

Liquid AI

45,893 subscribers

42,403 次观看 • 1 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

8 条评论

Dorian 的头像

Dorian1 个月前

@maximelabonne @EdoardMosca I love liquid models ❤️

Liquid AI 的头像

Liquid AI1 个月前

@maximelabonne @EdoardMosca ❤️

Javier • priv/acc 的头像

Javier • priv/acc1 个月前

@maximelabonne @EdoardMosca Sound like you are doing the same I'm willing with my model Javi 🔥 following your work closely! 👏👏

Nick Emb 的头像

Nick Emb1 个月前

@maximelabonne @EdoardMosca Liquid gold!

Kawai 的头像

Kawai1 个月前

@maximelabonne @EdoardMosca woah cool

Harley Lewis Foote 的头像

Harley Lewis Foote1 个月前

@maximelabonne @EdoardMosca Post-training is where it stops being a chatbot and starts being reliable.

Fajar M Reza 的头像

Fajar M Reza1 个月前

@maximelabonne @EdoardMosca Useful on-device agents need robust tool use, longer context, and recovery.

Marko Tasic 的头像

Marko Tasic1 个月前

@maximelabonne @EdoardMosca I love this kind of videos

相关视频

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Liquid AI

26,121 次观看 • 5 个月前

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

126,309 次观看 • 1 年前

Liquid AI's head of post-training explained how they built a small model that runs on-device under 1 GB in 20 minutes - better than $2500 small-model bootcamps. pick LFM2.5 base -> on-policy preference alignment -> agentic reinforcement learning -> curriculum training -> iterative model merging -> ship a 1B model that reliably calls tools on your phone. That loop is why frontier small models now beat 70B models on the tasks that actually matter. LFM2.5 + on-policy DPO + agentic RL + curriculum training + iterative merging - that's the stack. Watch and save it, then run a 1B agent on your phone tonight.

Liquid AI's head of post-training explained how they built a small model that runs on-device under 1 GB in 20 minutes - better than $2500 small-model bootcamps. pick LFM2.5 base -> on-policy preference alignment -> agentic reinforcement learning -> curriculum training -> iterative model merging -> ship a 1B model that reliably calls tools on your phone. That loop is why frontier small models now beat 70B models on the tasks that actually matter. LFM2.5 + on-policy DPO + agentic RL + curriculum training + iterative merging - that's the stack. Watch and save it, then run a 1B agent on your phone tonight.

h100envy

87,975 次观看 • 3 个月前

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

DeepLearning.AI

16,779 次观看 • 1 年前

Women get MILITARY TRAINING under 'Devoted to Iran' campaign Volunteers learn how to handle weapons and carry out basic military tasks

Women get MILITARY TRAINING under 'Devoted to Iran' campaign Volunteers learn how to handle weapons and carry out basic military tasks

RTVisual

11,747 次观看 • 1 个月前

GLM 5.3’s biggest leap came from post-training alone. “GLM 5.3 was built entirely by post-training the previous model. For me, that was a wake-up call.” “Post-training is far more GPU-efficient than pre-training from scratch. Plenty of companies can’t afford to train a model from scratch, but they have proprietary data. They can post-train themselves and beat some of the closed models.” “There’s still so much juice left to squeeze from post-training. We compared GLM 5.3’s cyber capabilities with 5.2, and it’s a night and day difference. That’s all just post-training.”

GLM 5.3’s biggest leap came from post-training alone. “GLM 5.3 was built entirely by post-training the previous model. For me, that was a wake-up call.” “Post-training is far more GPU-efficient than pre-training from scratch. Plenty of companies can’t afford to train a model from scratch, but they have proprietary data. They can post-train themselves and beat some of the closed models.” “There’s still so much juice left to squeeze from post-training. We compared GLM 5.3’s cyber capabilities with 5.2, and it’s a night and day difference. That’s all just post-training.”

SemiAnalysis

214,839 次观看 • 20 天前

Google's Jeff Dean says current pre-training is passive: initialize a model, stream the internet past it, let it observe But models need to learn not just from data, but by acting, predicting, and choosing what to learn from next "we have this artificial distinction now between pre and post-training, and it shouldn't exist long term"

Google's Jeff Dean says current pre-training is passive: initialize a model, stream the internet past it, let it observe But models need to learn not just from data, but by acting, predicting, and choosing what to learn from next "we have this artificial distinction now between pre and post-training, and it shouldn't exist long term"

Haider.

55,812 次观看 • 7 个月前

An exciting new course: Fine-tuning and Reinforcement Learning for LLMs: Intro to Post-training, taught by Sharon Zhou, VP of AI at AMD. Available now at Post-training is the key technique used by frontier labs to turn a base LLM--a model trained on massive unlabeled text to predict the next word/token--into a helpful, reliable assistant that can follow instructions. I've also seen many applications where post-training is what turns a demo application that works only 80% of the time into a reliable system that consistently performs. This course will teach you the most important post-training techniques! In this 5 module course, Sharon walks you through the complete post-training pipeline: supervised fine-tuning, reward modeling, RLHF, and techniques like PPO and GRPO. You'll also learn to use LoRA for efficient training, and to design evals that catch problems before and after deployment. Skills you'll gain: - Apply supervised fine-tuning and reinforcement learning (RLHF, PPO, GRPO) to align models to desired behaviors - Use LoRA for efficient fine-tuning without retraining entire models - Prepare datasets and generate synthetic data for post-training - Understand how to operate LLM production pipelines, with go/no-go decision points and feedback loops These advanced methods aren’t limited to frontier AI labs anymore, and you can now use them in your own applications. Learn here:

An exciting new course: Fine-tuning and Reinforcement Learning for LLMs: Intro to Post-training, taught by Sharon Zhou, VP of AI at AMD. Available now at Post-training is the key technique used by frontier labs to turn a base LLM--a model trained on massive unlabeled text to predict the next word/token--into a helpful, reliable assistant that can follow instructions. I've also seen many applications where post-training is what turns a demo application that works only 80% of the time into a reliable system that consistently performs. This course will teach you the most important post-training techniques! In this 5 module course, Sharon walks you through the complete post-training pipeline: supervised fine-tuning, reward modeling, RLHF, and techniques like PPO and GRPO. You'll also learn to use LoRA for efficient training, and to design evals that catch problems before and after deployment. Skills you'll gain: - Apply supervised fine-tuning and reinforcement learning (RLHF, PPO, GRPO) to align models to desired behaviors - Use LoRA for efficient fine-tuning without retraining entire models - Prepare datasets and generate synthetic data for post-training - Understand how to operate LLM production pipelines, with go/no-go decision points and feedback loops These advanced methods aren’t limited to frontier AI labs anymore, and you can now use them in your own applications. Learn here:

Andrew Ng

133,386 次观看 • 1 年前

JARVIS-VLA just dropped on Hugging Face Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse obtain VLA models in Minecraft that can follow human instructions on over 1k different atomic tasks, including crafting, smelting, cooking, mining, and killing. experiments demonstrate that post-training on non-trajectory tasks leads to a significant 40% improvement over the best agent baseline on a diverse set of atomic tasks. Furthermore, demonstrate that approach surpasses traditional imitation learning-based policies in Minecraft, achieving state-of-the-art performance.

JARVIS-VLA just dropped on Hugging Face Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse obtain VLA models in Minecraft that can follow human instructions on over 1k different atomic tasks, including crafting, smelting, cooking, mining, and killing. experiments demonstrate that post-training on non-trajectory tasks leads to a significant 40% improvement over the best agent baseline on a diverse set of atomic tasks. Furthermore, demonstrate that approach surpasses traditional imitation learning-based policies in Minecraft, achieving state-of-the-art performance.

AK

60,354 次观看 • 1 年前

You give an AI agent a task. What happens next? This video walks through how agents use models and tools to complete tasks, and what makes them fast and responsive. No technical background needed.

You give an AI agent a task. What happens next? This video walks through how agents use models and tools to complete tasks, and what makes them fast and responsive. No technical background needed.

NVIDIA AI

12,972 次观看 • 23 天前

When should you start post-training your own models? Fireworks CEO Lin Qiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our Sequoia Capital "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training

When should you start post-training your own models? Fireworks CEO Lin Qiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our Sequoia Capital "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training

Sonya Huang 🐥

28,212 次观看 • 2 个月前

It’s time to level up your agentic development with Android skills. 🚀 These AI-optimized instructions automate more complex tasks like XML-to-Compose and AGP9 migrations, follow our official best practices and guidance on Android development, and are activated on demand by AI tools. Get started with Android skills →

It’s time to level up your agentic development with Android skills. 🚀 These AI-optimized instructions automate more complex tasks like XML-to-Compose and AGP9 migrations, follow our official best practices and guidance on Android development, and are activated on demand by AI tools. Get started with Android skills →

Android Developers

15,758 次观看 • 6 个月前

1/ Six models, from 0.9B to 375B total parameters, released as one connected fleet. Institute of Foundation Models’s K2 Horizon is ambitious because it tries to make the training process inspectable, from pretraining to reasoning and agentic post-training. A closer look:

1/ Six models, from 0.9B to 375B total parameters, released as one connected fleet. Institute of Foundation Models’s K2 Horizon is ambitious because it tries to make the training process inspectable, from pretraining to reasoning and agentic post-training. A closer look:

Chubby♨️

22,026 次观看 • 1 个月前

Introducing NVIDIA Nemotron 3 Ultra. A frontier smart open model built for long-running agents that need to plan, reason, use tools and keep working across complex coding, research and enterprise workflows. Up to 5x faster inference and up to 30% lower cost for agentic tasks. Learn more:

Introducing NVIDIA Nemotron 3 Ultra. A frontier smart open model built for long-running agents that need to plan, reason, use tools and keep working across complex coding, research and enterprise workflows. Up to 5x faster inference and up to 30% lower cost for agentic tasks. Learn more:

NVIDIA

230,981 次观看 • 5 个月前

Gemma and the power of post-training. 💪 Google DeepMind’s Léonard Hussenot details: 🔍 How Gemma facilitates post-training research 🚀 Best-of-N Distillation (BOND) for model efficiency 🏆 Improving reward models with weight averaging 👀 →

Gemma and the power of post-training. 💪 Google DeepMind’s Léonard Hussenot details: 🔍 How Gemma facilitates post-training research 🚀 Best-of-N Distillation (BOND) for model efficiency 🏆 Improving reward models with weight averaging 👀 →

Google for Developers

18,684 次观看 • 1 年前

I like to think of evals as something active, not passive -- it's a North Star that steers LLMs toward higher intelligence. Evals should drive your RL/SFT/post-training decisions. Internal evals at frontier labs make a huge difference -- and you can see it in how models behave differently (GPT seems better at one-shot tasks, Claude at multi-turn). If you want to learn more about building evals that actually improve your model in post-training, check out our AMD x DeeplearningAI course "Fine-tuning & RL for LLMs: Intro to Post-training" (content is free):

I like to think of evals as something active, not passive -- it's a North Star that steers LLMs toward higher intelligence. Evals should drive your RL/SFT/post-training decisions. Internal evals at frontier labs make a huge difference -- and you can see it in how models behave differently (GPT seems better at one-shot tasks, Claude at multi-turn). If you want to learn more about building evals that actually improve your model in post-training, check out our AMD x DeeplearningAI course "Fine-tuning & RL for LLMs: Intro to Post-training" (content is free):

Sharon Zhou

16,903 次观看 • 9 个月前

Hybrid models efficiently route complex tasks to larger cloud models, but how do you go about creating one? 🤔 Cactus solved this by post-training Gemma 4 to return a confidence score for every prompt. Process the prompt locally when confidence is high, route to a larger model when confidence is low. Routing only happens 15%-55% of time based on domain benchmarks.

Hybrid models efficiently route complex tasks to larger cloud models, but how do you go about creating one? 🤔 Cactus solved this by post-training Gemma 4 to return a confidence score for every prompt. Process the prompt locally when confidence is high, route to a larger model when confidence is low. Routing only happens 15%-55% of time based on domain benchmarks.

Google Gemma

25,222 次观看 • 3 个月前

DeepSeek-V4 dropped. 1M context. 10x smaller KV cache. First open model where the context window and the agentic post-training meet.

DeepSeek-V4 dropped. 1M context. 10x smaller KV cache. First open model where the context window and the agentic post-training meet.

Ben Burtenshaw

49,900 次观看 • 6 个月前