Загрузка видео...

Не удалось загрузить видео

На главную

3Blue1Brown’s new video explains why every LLM is actually a compression machine. everyone describes pre-training as “next token prediction” but that’s just the surface-level objective. in reality it is a means to making the most efficient text compressor. prediction and compression are two sides of the same coin. when...

119,751 просмотров • 1 месяц назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

.Naval: Look at reality the way it is. Try to take yourself out of the equation. “Labels like pessimist, optimist, cynic, introvert, extrovert—these are very self limiting. Humans are very dynamic. There are times when you feel like being introverted. There are times when you feel like being extroverted. There are contexts in which you’ll be pessimistic. There are contexts in which you’ll be optimistic. Leave all those labels alone. It’s better just to look at the problem at hand. Look at reality the way it is. Try to take yourself out of the equation, in a sense. Obviously you’re involved. But motivated reasoning is the worst kind of reasoning. You’re not going to find truth through highly motivated reasoning. You have to be objective. And objective means trying to take yourself out of it as much as possible, or at least your personality out of it as much as possible. And so to the extent you run with this thick identity and personality, it’s going to cloud your judgment. It’s going to try and lock you into the past. If you say, ‘I’m a depressed, unhappy person—I’m going to be unhappy.’ That’s a way of locking yourself into your past. Even saying, ‘I have trauma, I have PTSD,’ yeah, you feel something—there are memories, there are flashes, there are occasional bad feelings. But don’t define yourself by it because then you’ll lock it into your identity and you’re just going to loop on it. It’s better to stay flexible because reality is always changing and you have to be able to adapt to it. Adaptation is also intelligence, adaptation is survival. Adaptation is kind of how you’re here. You’re here because you’re an adapter and your ancestors were adapters. So to adapt you’ll have to see things clearly and if you’re seeing them through your own identity, it’s going to cloud your judgment.”

Arjun Khemani

251,772 просмотров • 1 год назад

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 просмотров • 1 год назад