Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

A science of post-training can be built on interpretability. Models know when they reward hack, and we see it, yet training ignores it. We propose Mechanistic Shaping: using the model’s own concepts to write training rules. In vision, DPO & RL, and its effects on monitoring 🧵

Thomas Fel

3,716 subscribers

41,904 görüntüleme • 1 gün önce •via X (Twitter)

Bilim & Teknoloji Eğitim

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

New work on Training Foundation Models with a Single Algorithm: Paper: Code, Models: This work introduces Unified Training, a framework that unifies pretraining, midtraining, supervised fine-tuning and RL under a single maximum-likelihood objective. Building on MaxRL, it enables static data and RL-generated data to be trained together within the same system. Empirically, Unified Training improves accuracy and compute efficiency, generalizes better to harder reasoning problems, and reduces forgetting. Interestingly, reusing data across training stages can improve performance even when conventional staged training fails. It also produces models that adapt more effectively to new tasks, leading to stronger continual learning. See an excellent thread by Yuda Song.

New work on Training Foundation Models with a Single Algorithm: Paper: Code, Models: This work introduces Unified Training, a framework that unifies pretraining, midtraining, supervised fine-tuning and RL under a single maximum-likelihood objective. Building on MaxRL, it enables static data and RL-generated data to be trained together within the same system. Empirically, Unified Training improves accuracy and compute efficiency, generalizes better to harder reasoning problems, and reduces forgetting. Interestingly, reusing data across training stages can improve performance even when conventional staged training fails. It also produces models that adapt more effectively to new tasks, leading to stronger continual learning. See an excellent thread by Yuda Song.

Russ Salakhutdinov

42,317 görüntüleme • 7 gün önce

Harness adapters training is now live! We are training LoRA adapters on LFM2.5 / LFM3 from Liquid AI and the recent Twil-9B model from webAI using RL to make them more efficient and accurate on agentic tasks! You can follow the training going live here:

Harness adapters training is now live! We are training LoRA adapters on LFM2.5 / LFM3 from Liquid AI and the recent Twil-9B model from webAI using RL to make them more efficient and accurate on agentic tasks! You can follow the training going live here:

younes

33,870 görüntüleme • 12 saat önce

As a supporter of the open weights ecosystem, we're proud to be a post-training partner for NVIDIA Nemotron. We post-train Nemotron models for customer use cases, de-risk mainline RL runs on our AC2 platform and training stack, and contribute aggregate workload statistics for inference benchmarking. This is how open models get better, and we're excited to keep working closely with NVIDIA AI.

As a supporter of the open weights ecosystem, we're proud to be a post-training partner for NVIDIA Nemotron. We post-train Nemotron models for customer use cases, de-risk mainline RL runs on our AC2 platform and training stack, and contribute aggregate workload statistics for inference benchmarking. This is how open models get better, and we're excited to keep working closely with NVIDIA AI.

Applied Compute

25,683 görüntüleme • 3 ay önce

Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive. So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.

Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive. So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.

Modal

167,578 görüntüleme • 24 gün önce

Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!

Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!

Jake Ward

304,245 görüntüleme • 18 gün önce

1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky! Very excited to share our new paper: Training Language Models to Fail Gracefully 🧵

1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky! Very excited to share our new paper: Training Language Models to Fail Gracefully 🧵

Daman Arora

59,926 görüntüleme • 21 gün önce

I don’t know who needs to hear this but…. If you’re using mostly free weights and very few well-built machines in your training, you are FOR SURE costing yourself gains If you’re using a blend of free weights and well-built machines in your training, you’re PROBABLY doing a pretty good job with exercise selection but you may be leaving SOME gains on the table If you’re using mostly well-built machines and very few free weights in your training, you are very likely putting yourself in a position to MAXIMIZE gains I DO NOT MAKE THE RULES!

I don’t know who needs to hear this but…. If you’re using mostly free weights and very few well-built machines in your training, you are FOR SURE costing yourself gains If you’re using a blend of free weights and well-built machines in your training, you’re PROBABLY doing a pretty good job with exercise selection but you may be leaving SOME gains on the table If you’re using mostly well-built machines and very few free weights in your training, you are very likely putting yourself in a position to MAXIMIZE gains I DO NOT MAKE THE RULES!

Dean Turner

136,355 görüntüleme • 1 ay önce

We barely made it to training. We had heavy hail and rain, but don't be sad.

We barely made it to training. We had heavy hail and rain, but don't be sad.

hay nyash

61,485 görüntüleme • 3 ay önce

New work: The Value Axis 🎯 How do LLMs choose which path to take mid-task? We find they internally track the chance of reaching their goal along a linear axis, akin to a value function in RL. We show it modulates confidence in math & coding and can be reshaped with DPO and SFT.

New work: The Value Axis 🎯 How do LLMs choose which path to take mid-task? We find they internally track the chance of reaching their goal along a linear axis, akin to a value function in RL. We show it modulates confidence in math & coding and can be reshaped with DPO and SFT.

Nick Jiang

29,777 görüntüleme • 4 ay önce

Introducing autoresearch for arXiv papers: replicate and experiment on any arXiv paper using an army of Claude/Codex agents For post-training, your agents can use Tinker to launch concurrent RL runs while you watch your experiment tree grow Try it out now on OpenResearch:

Introducing autoresearch for arXiv papers: replicate and experiment on any arXiv paper using an army of Claude/Codex agents For post-training, your agents can use Tinker to launch concurrent RL runs while you watch your experiment tree grow Try it out now on OpenResearch:

alphaXiv

61,732 görüntüleme • 2 ay önce

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

Aayush Karan

280,889 görüntüleme • 1 yıl önce

What happens when we lose an engine after decision speed? We go. We’re committed. We can’t stop on the runway. One of the toughest maneuvers in aviation and one we practice ad nauseum in training. I can do it in my sleep. Which is a good thing. Rest assured your crew can do it too. 🔥

What happens when we lose an engine after decision speed? We go. We’re committed. We can’t stop on the runway. One of the toughest maneuvers in aviation and one we practice ad nauseum in training. I can do it in my sleep. Which is a good thing. Rest assured your crew can do it too. 🔥

Buzz Patterson

42,841 görüntüleme • 1 ay önce

Models usually know when they’re doing something wrong. Probes let us “read their minds” to detect offensive cyber intent, reward hacking, strategic deception, and more. If you build or serve models, they’re your first line of defense. Our latest post explains how they work & how to use them - the first of a series of educational posts on applied AI interpretability. 🧵

Models usually know when they’re doing something wrong. Probes let us “read their minds” to detect offensive cyber intent, reward hacking, strategic deception, and more. If you build or serve models, they’re your first line of defense. Our latest post explains how they work & how to use them - the first of a series of educational posts on applied AI interpretability. 🧵

Goodfire

31,225 görüntüleme • 1 ay önce

We built an AI model to simulate how a fruit fly walks, flies and behaves – in partnership with HHMI | Janelia. 🪰 Our computerized insect replicates realistic motion, and can even use its eyes to control its actions. Here’s how we developed it – and what it means for science. 🧵

We built an AI model to simulate how a fruit fly walks, flies and behaves – in partnership with HHMI | Janelia. 🪰 Our computerized insect replicates realistic motion, and can even use its eyes to control its actions. Here’s how we developed it – and what it means for science. 🧵

Google DeepMind

1,060,148 görüntüleme • 1 yıl önce

Monfils : « Im training hard on clay to finish it well in Paris » The training session :

Monfils : « Im training hard on clay to finish it well in Paris » The training session :

Regista Betting

111,873 görüntüleme • 5 ay önce

ChatGPT uses your data for training its Large Language Model (LLM). But now you can ask ChatGPT to NOT use data for training its model. Go to "Settings." Click on "Show" in front of "Data Controls." Turn off "Chat History and Training."

ChatGPT uses your data for training its Large Language Model (LLM). But now you can ask ChatGPT to NOT use data for training its model. Go to "Settings." Click on "Show" in front of "Data Controls." Turn off "Chat History and Training."

Mushtaq Bilal, PhD

68,006 görüntüleme • 3 yıl önce

Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog:

Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog:

Sophie Wang

198,863 görüntüleme • 1 ay önce

In decentralized training, every merge stops the whole swarm from training while it waits on the slowest member. With our latest update, merging is ~5x faster and training is almost 20% cheaper. The biggest unlock was to convert the network graph into a tree during merging. 🧵 1/(n)

In decentralized training, every merge stops the whole swarm from training while it waits on the slowest member. With our latest update, merging is ~5x faster and training is almost 20% cheaper. The biggest unlock was to convert the network graph into a tree during merging. 🧵 1/(n)

IOTA ・ SN9

15,432 görüntüleme • 18 gün önce

World BanksPi developer and Korea Ecosystem Ambassador Carter Lim conducted 4 hours of training in Busan on the importance of ecosystem development, various tasks to prepare for the upcoming open mainnet, and the concept of GCV. We also conducted training on Dex, token, and equity pool. In the future, he will travel around the country to conduct such training.

World BanksPi developer and Korea Ecosystem Ambassador Carter Lim conducted 4 hours of training in Busan on the importance of ecosystem development, various tasks to prepare for the upcoming open mainnet, and the concept of GCV. We also conducted training on Dex, token, and equity pool. In the future, he will travel around the country to conduct such training.

jintaek JUN

17,857 görüntüleme • 9 ay önce