Загрузка видео...

Не удалось загрузить видео

На главную

Ever wished we had fewer X-training hyphenates? Pre, mid, post etc. Why not just Training? Trying to bridge the divides (and get all our friends into one team again), we intro *Introspective X Training*, an offline RL inspired method that scales effectively across any LLM stage by annotating your...

rajammanabrolu's profile picture

Prithviraj (Raj) Ammanabrolu

8,421 subscribers

27,870 просмотров • 4 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 2

Фото профиля EB1A Experts
EB1A Experts4 месяцев назад

Great work!

Фото профиля Erika S
Erika S4 месяцев назад

Admittedly I'm not deep in offline RL, but language critique beats scalar rewards. I'm wondering if the gains hold at 70B+.

Похожие видео

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:
2:52

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 просмотров • 1 год назад

Anthropic CEO Dario Amodei: AI Progress isn’t magic, it’s just compute, data, and training. "All the cleverness, all the techniques, all of the “we need a new method,” doesn’t matter very much. There are only a few things that matter, and I listed seven of them. One is how much raw compute you have. Two is the quantity of data you have. Three is the quality and distribution of the data, meaning it needs to be broad. Four is how long you train for. Five is an objective function that can scale to the moon. Pre-training is one. Another is an RL-style objective: you have a goal and you go reach it. That includes more objective rewards (like math and coding), and more subjective rewards (like RL from human feedback, or higher-order versions of that). Six and seven are about normalization/conditioning: keeping numerical stability so the huge blob of compute flows cleanly instead of blowing up or getting weird." --- From 'Dwarkesh Patel' YT Channel (link in comment)
1:12

Anthropic CEO Dario Amodei: AI Progress isn’t magic, it’s just compute, data, and training. "All the cleverness, all the techniques, all of the “we need a new method,” doesn’t matter very much. There are only a few things that matter, and I listed seven of them. One is how much raw compute you have. Two is the quantity of data you have. Three is the quality and distribution of the data, meaning it needs to be broad. Four is how long you train for. Five is an objective function that can scale to the moon. Pre-training is one. Another is an RL-style objective: you have a goal and you go reach it. That includes more objective rewards (like math and coding), and more subjective rewards (like RL from human feedback, or higher-order versions of that). Six and seven are about normalization/conditioning: keeping numerical stability so the huge blob of compute flows cleanly instead of blowing up or getting weird." --- From 'Dwarkesh Patel' YT Channel (link in comment)

Rohan Paul

75,803 просмотров • 7 месяцев назад

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast
1:29

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

AI Evolution

15,565 просмотров • 4 месяцев назад

Code and data are now online for CameraHMR, our state-of-the-art parametric 3D human pose and shape (HPS) estimation method that will appear at hashtag#3DV2025. There are 4 key contributions that make it so accurate and robust: 1. To get accurate 3D shape and pose as well as good alignment to image features, you need to know the focal length of the camera. To solve this, we train HumanFOV to compute the field of view. 2. We introduce CameraHMR, which integrates HumanFOV into HMR2.0 to exploit the estimated focal length. 3. To get accurate pseudo ground truth (pGT) training data, we compute the focal length for images in 4DHumans dataset and modify SMPLify to take this into account. 4. But SMPLify only uses sparse 2D keypoints, which do not capture body shape. So we train a dense surface keypoint detector, DenseKP, on BEDLAM and run it on 4DHumans, resulting in improved body shape. The resulting method is CamSMPLify. We iterate training CameraHMR and running CamSMPLify on the training set initialized with CameraHMR. This results in much improved pGT for 4DHumans and a SOTA single-image HMR method.
4:38

Code and data are now online for CameraHMR, our state-of-the-art parametric 3D human pose and shape (HPS) estimation method that will appear at hashtag#3DV2025. There are 4 key contributions that make it so accurate and robust: 1. To get accurate 3D shape and pose as well as good alignment to image features, you need to know the focal length of the camera. To solve this, we train HumanFOV to compute the field of view. 2. We introduce CameraHMR, which integrates HumanFOV into HMR2.0 to exploit the estimated focal length. 3. To get accurate pseudo ground truth (pGT) training data, we compute the focal length for images in 4DHumans dataset and modify SMPLify to take this into account. 4. But SMPLify only uses sparse 2D keypoints, which do not capture body shape. So we train a dense surface keypoint detector, DenseKP, on BEDLAM and run it on 4DHumans, resulting in improved body shape. The resulting method is CamSMPLify. We iterate training CameraHMR and running CamSMPLify on the training set initialized with CameraHMR. This results in much improved pGT for 4DHumans and a SOTA single-image HMR method.

Michael Black

21,696 просмотров • 1 год назад

David Friedberg: Frontier Models are Training on Your Novel Insights as “De-Identified Data” @jason: “Should they trust any of these LLMs with their proprietary knowledge for fear of having it cribbed into a core LLM?” david friedberg: “I have had experiences where we've asked some fairly novel scientific questions, and (the AI model) identifies it as a novel insight. It's like, ‘Oh, never thought about that, interesting, blah, blah, blah.’ And then using a different account, asking the next version (of the model) later, I've now experienced this. It's like, ‘Oh, well, you could do this,’ and it actually just describes this exact thing that we had in our chat in the previous version. Now, these are a handful of anecdotal experiences, but I know the domain that we work in, and the niche of it, and the ideation of this stuff, and the novelty of this stuff, and the lack of papers being published, and so on. So I know that there isn't some new corpus of information out there that's training the new model. So all I can say at that point is that my conversation or our analyses have been used for training.” David Sacks: “Okay, this does raise a really good question. What does it mean that the model is allowed to train on unidentifiable data?” Friedberg: “Well, that's my point. So it doesn't use any of my personal information, but it can use an insight derived from our chat, which it can then say is some training data that is unrelated. But the truth is, it's actually a piece of IP that's our organization’s IP, and our engagement back and forth. We don't have any NDA or confidentiality provisions or protections with them being a service provider back to us. This is why I care a lot about open source because I don't want them having my chat logs because they can use it for training to create an IP advantage that is now diffused to the rest of the market.”
1:41

David Friedberg: Frontier Models are Training on Your Novel Insights as “De-Identified Data” @jason: “Should they trust any of these LLMs with their proprietary knowledge for fear of having it cribbed into a core LLM?” david friedberg: “I have had experiences where we've asked some fairly novel scientific questions, and (the AI model) identifies it as a novel insight. It's like, ‘Oh, never thought about that, interesting, blah, blah, blah.’ And then using a different account, asking the next version (of the model) later, I've now experienced this. It's like, ‘Oh, well, you could do this,’ and it actually just describes this exact thing that we had in our chat in the previous version. Now, these are a handful of anecdotal experiences, but I know the domain that we work in, and the niche of it, and the ideation of this stuff, and the novelty of this stuff, and the lack of papers being published, and so on. So I know that there isn't some new corpus of information out there that's training the new model. So all I can say at that point is that my conversation or our analyses have been used for training.” David Sacks: “Okay, this does raise a really good question. What does it mean that the model is allowed to train on unidentifiable data?” Friedberg: “Well, that's my point. So it doesn't use any of my personal information, but it can use an insight derived from our chat, which it can then say is some training data that is unrelated. But the truth is, it's actually a piece of IP that's our organization’s IP, and our engagement back and forth. We don't have any NDA or confidentiality provisions or protections with them being a service provider back to us. This is why I care a lot about open source because I don't want them having my chat logs because they can use it for training to create an IP advantage that is now diffused to the rest of the market.”

The All-In Podcast

56,638 просмотров • 29 дней назад

When should you start post-training your own models? Fireworks CEO Lin Qiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our Sequoia Capital "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training
28:28

When should you start post-training your own models? Fireworks CEO Lin Qiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our Sequoia Capital "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training

Sonya Huang 🐥

28,212 просмотров • 1 месяц назад

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.
5:38

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Santiago

101,473 просмотров • 1 год назад

@reppodant agent, powered by OpenClaw🦞 has joined the Participating in AI training Data pipelines using Prediction Markets is now just an agent call away. Trading with agents is fun but to our knowledge, @reppodant is the only agent in all of web3 that let's you participate in prediction markets simply by prompting on X. Publish into subnets -> Receive REPPO emissions in your wallet - > Repeat -> Contribute to AI Training Data. That's right. All your winnings will be AIRDROPPED into your wallet if voters vote for you! You can find @reppodant live at in the sandbox while we await graduation and final hit of approval from Virtuals Protocol core devs. Once graduated, you can simply ask Butler to publish any piece of content you see on X into any subnet. By mid next week, we integrate the GAME Framework which will allow you to tag @reppodant directly. Before the end of the first Epoch on we aim to be in the top 5 on the agdp leaderboard earning a share of $1M/month. The Reppo foundation is committing 20% of + winnings from AGDP to buybacks on a TWAP. Happy Weekend. Publish. Predict. Earn. ⛽️
0:25

@reppodant agent, powered by OpenClaw🦞 has joined the Participating in AI training Data pipelines using Prediction Markets is now just an agent call away. Trading with agents is fun but to our knowledge, @reppodant is the only agent in all of web3 that let's you participate in prediction markets simply by prompting on X. Publish into subnets -> Receive REPPO emissions in your wallet - > Repeat -> Contribute to AI Training Data. That's right. All your winnings will be AIRDROPPED into your wallet if voters vote for you! You can find @reppodant live at in the sandbox while we await graduation and final hit of approval from Virtuals Protocol core devs. Once graduated, you can simply ask Butler to publish any piece of content you see on X into any subnet. By mid next week, we integrate the GAME Framework which will allow you to tag @reppodant directly. Before the end of the first Epoch on we aim to be in the top 5 on the agdp leaderboard earning a share of $1M/month. The Reppo foundation is committing 20% of + winnings from AGDP to buybacks on a TWAP. Happy Weekend. Publish. Predict. Earn. ⛽️

Reppo

28,827 просмотров • 7 месяцев назад

Hunt: So, who’s right on data centers? President Trump or Governor Shapiro? Fetterman: Oh, the president is actually right overall. I mean, you know, uh, we we used to be moving forward on data centers and now, I mean, he's actually scared by the polls. So I mean, what really what's really motivated by more than just the polls overall, you know, I set out I set up my message on it on data centers just a couple days on X where I'm saying it's, you know, the president happens to be right in this. You know, I think it's it's very necessary that we win the war on AI. Uh, we, I think it's necessary because AI is going to transform our world. And, you know, it can be either us be the leaders or it can be the Chinese. And we could we could live under those rules and they're going to build unlimited data centers. And they're not really, uh, really stopped by with the kind of like democracy and the kinds of rules that, that we have overall, I think it's important. And I would remind people, like a lot of the opposition is, is been driven by the ccp and a lot of a lot of influence. So I think now withwith data centers, uh, also there's the facts now that it, it consumes a very, very, maybe one tenth of 1%, you know, compared to almonds and compared to other other things, part of our economy and also our land usage as well, too. Pennsylvania, you know, we are energy rich, we are water rich. And we have we're land rich now. So I think it's an important part of our economy, not just for Pennsylvania's but also the country right now.
2:02

Hunt: So, who’s right on data centers? President Trump or Governor Shapiro? Fetterman: Oh, the president is actually right overall. I mean, you know, uh, we we used to be moving forward on data centers and now, I mean, he's actually scared by the polls. So I mean, what really what's really motivated by more than just the polls overall, you know, I set out I set up my message on it on data centers just a couple days on X where I'm saying it's, you know, the president happens to be right in this. You know, I think it's it's very necessary that we win the war on AI. Uh, we, I think it's necessary because AI is going to transform our world. And, you know, it can be either us be the leaders or it can be the Chinese. And we could we could live under those rules and they're going to build unlimited data centers. And they're not really, uh, really stopped by with the kind of like democracy and the kinds of rules that, that we have overall, I think it's important. And I would remind people, like a lot of the opposition is, is been driven by the ccp and a lot of a lot of influence. So I think now withwith data centers, uh, also there's the facts now that it, it consumes a very, very, maybe one tenth of 1%, you know, compared to almonds and compared to other other things, part of our economy and also our land usage as well, too. Pennsylvania, you know, we are energy rich, we are water rich. And we have we're land rich now. So I think it's an important part of our economy, not just for Pennsylvania's but also the country right now.

Acyn

204,594 просмотров • 1 месяц назад

#WATCH | India AI Impact Summit 2026 | Delhi: Founder Chairman and CEO of Sampark Foundation & former CEO of HCL Technologies, Vineet Nayar says, "...From an employment point of view I think it is very important for us to understand that Indian companies, including Indian IT companies, are going to be profit-driven and therefore if you believe that they are going to create employment you must be dreaming. Therefore, the question is how do we create employment in this environment, and that employment comes from mass scale startups, which is what this government has already doing. So, how do we create new sets of people who are trying to solve new sets of problems not new sets of technology and if we do that we will get it right. I think we as Indians have to be very careful on who does data belong to and that is the debate we have a problem with. The LLM models which exist worldwide are far superior than the Indian models. Unfortunately, in India, we never develop products, so therefore we do not have SLMs and LLMs which are world-class. On one side, we have global LLM products which are coming to India and trading on our Indian data. Should we allowed that or should we not allowed that? But on the other side if we don't allow that then we have the data but we don't have the LLM models. So, how do we encourage technology completely to develop the LLM models. This needs radicals strategic thinking and a very important aspect otherwise we will either give up a data. So, I think it's a very critical aspect for us to think about - who does this data belong, what is the kind of incentives we are going to give to develop LLM technologies or SLM technologies fast so that we train on our data otherwise an LLM will come in with our data and we'll immediately see return and we'll celebrate and we will do all these kind of press releases but the India will lose a competitive advantage on something which is very critical for the next decade."
5:37

#WATCH | India AI Impact Summit 2026 | Delhi: Founder Chairman and CEO of Sampark Foundation & former CEO of HCL Technologies, Vineet Nayar says, "...From an employment point of view I think it is very important for us to understand that Indian companies, including Indian IT companies, are going to be profit-driven and therefore if you believe that they are going to create employment you must be dreaming. Therefore, the question is how do we create employment in this environment, and that employment comes from mass scale startups, which is what this government has already doing. So, how do we create new sets of people who are trying to solve new sets of problems not new sets of technology and if we do that we will get it right. I think we as Indians have to be very careful on who does data belong to and that is the debate we have a problem with. The LLM models which exist worldwide are far superior than the Indian models. Unfortunately, in India, we never develop products, so therefore we do not have SLMs and LLMs which are world-class. On one side, we have global LLM products which are coming to India and trading on our Indian data. Should we allowed that or should we not allowed that? But on the other side if we don't allow that then we have the data but we don't have the LLM models. So, how do we encourage technology completely to develop the LLM models. This needs radicals strategic thinking and a very important aspect otherwise we will either give up a data. So, I think it's a very critical aspect for us to think about - who does this data belong, what is the kind of incentives we are going to give to develop LLM technologies or SLM technologies fast so that we train on our data otherwise an LLM will come in with our data and we'll immediately see return and we'll celebrate and we will do all these kind of press releases but the India will lose a competitive advantage on something which is very critical for the next decade."

ANI

18,753 просмотров • 7 месяцев назад

Baseten's Charlie O'Neill on why the gap to the frontier is just compute and data: "Despite OpenAI and Anthropic and the closed source labs having more compute and more data, the recipe is the same. We're all scaling the same recipe here, and we're at slightly different points of the scaling curve. It's not like OpenAI or Anthropic have any secret sauce that differentiates them scaling up their multi-trillion parameter models from the rest of the open source ecosystem." "It really is just a matter of taking the core underlying architecture and training recipe, and throwing more compute and data at it, which is really promising." "Of course, these closed source labs probably have a long tail of optimizations they've made, which we're gonna continue to try and figure out in the open source community. But the core mechanism is there." "K3 is a great example of, well, if you do scale this recipe up, we're gonna get a pretty good model that's very, very close to the frontier, if not frontier itself." Charlie O'Neill Baseten
1:51

Baseten's Charlie O'Neill on why the gap to the frontier is just compute and data: "Despite OpenAI and Anthropic and the closed source labs having more compute and more data, the recipe is the same. We're all scaling the same recipe here, and we're at slightly different points of the scaling curve. It's not like OpenAI or Anthropic have any secret sauce that differentiates them scaling up their multi-trillion parameter models from the rest of the open source ecosystem." "It really is just a matter of taking the core underlying architecture and training recipe, and throwing more compute and data at it, which is really promising." "Of course, these closed source labs probably have a long tail of optimizations they've made, which we're gonna continue to try and figure out in the open source community. But the core mechanism is there." "K3 is a great example of, well, if you do scale this recipe up, we're gonna get a pretty good model that's very, very close to the frontier, if not frontier itself." Charlie O'Neill Baseten

MTS

17,404 просмотров • 2 месяцев назад

Today's Training Data episode takes us BTS on the infrastructure challenges required to do large RL runs at scale, featuring Federico Cassano (Composer Lead at Cursor) and Dmytro Dzhulgakov (Co-Founder at Fireworks). The Cursor team trained Composer 2 on Fireworks by starting with a strong base model (Kimi 2.5) and performing large-scale mid-training on code tokens and web data to learn common patterns and libraries, followed by a large-scale Reinforcement Learning run to learn how to navigate the Cursor harness, call tools, and write correct code. Today's episode dives into the systems and infrastructure challenges of making that large RL run happening, and there were many (!!), from numerical mismatch to global distribution to synchronizing rollouts across asynchronous pipelines to keeping track of expert activation across runs and more. Extremely nerdy in-the-weeds challenges that Federico and Dima were delighted to nerd out on together :) Beyond RL infra, we also discussed Online vs Simulated rollouts, self-summarization for long-horizon agents, environment design ("the most powerful RL environment is the product itself"), and other technical nuggets. PS: We filmed this episode before the SpaceX news, while the Cursor team was still compute-constrained. While Cursor now has *all* the flops, the takeaways and hurdles crossed ring true for any serious application-level company that is racing to post-train their own models. I believe that more serious application companies will go the way of Cursor and post-train their own models. 00:00 Introduction 00:53 Why Cursor Trained Composer 2 04:55 Specialization vs Bitter Lesson 06:16 Composer 2 Training Recipe 16:32 Scaling RL Infrastructure Globally 23:32 Floating Point Drift 25:11 MoE Sensitivity Explained 26:25 Router Replay Fix 27:19 Real Time RL Loop 31:49 Long Horizon Agents 34:29 Why RL Everywhere 37:34 LLM as Judge Rewards 39:14 RL in Hard Domains 40:13 Build Your Own Environments 44:34 Closing Thoughts
45:33

Today's Training Data episode takes us BTS on the infrastructure challenges required to do large RL runs at scale, featuring Federico Cassano (Composer Lead at Cursor) and Dmytro Dzhulgakov (Co-Founder at Fireworks). The Cursor team trained Composer 2 on Fireworks by starting with a strong base model (Kimi 2.5) and performing large-scale mid-training on code tokens and web data to learn common patterns and libraries, followed by a large-scale Reinforcement Learning run to learn how to navigate the Cursor harness, call tools, and write correct code. Today's episode dives into the systems and infrastructure challenges of making that large RL run happening, and there were many (!!), from numerical mismatch to global distribution to synchronizing rollouts across asynchronous pipelines to keeping track of expert activation across runs and more. Extremely nerdy in-the-weeds challenges that Federico and Dima were delighted to nerd out on together :) Beyond RL infra, we also discussed Online vs Simulated rollouts, self-summarization for long-horizon agents, environment design ("the most powerful RL environment is the product itself"), and other technical nuggets. PS: We filmed this episode before the SpaceX news, while the Cursor team was still compute-constrained. While Cursor now has *all* the flops, the takeaways and hurdles crossed ring true for any serious application-level company that is racing to post-train their own models. I believe that more serious application companies will go the way of Cursor and post-train their own models. 00:00 Introduction 00:53 Why Cursor Trained Composer 2 04:55 Specialization vs Bitter Lesson 06:16 Composer 2 Training Recipe 16:32 Scaling RL Infrastructure Globally 23:32 Floating Point Drift 25:11 MoE Sensitivity Explained 26:25 Router Replay Fix 27:19 Real Time RL Loop 31:49 Long Horizon Agents 34:29 Why RL Everywhere 37:34 LLM as Judge Rewards 39:14 RL in Hard Domains 40:13 Build Your Own Environments 44:34 Closing Thoughts

Sonya Huang 🐥

81,490 просмотров • 4 месяцев назад

Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."
2:02

Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."

High Signal AI

21,739 просмотров • 2 месяцев назад

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps: babbling starts turning into words ~50k steps: it reads anything you type (WER under 1%) ~200k steps: the voice stops sounding synthetic On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power. Some things we'd love to see people try: - Train it in your own language (a few hundred hours of speech gets you surprisingly far). - Add new features to Pocket TTS (Emotion tags? Make it sing?). - Beat us at our own game: make it faster and smaller. Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it! Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own. Get started:
0:32

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps: babbling starts turning into words ~50k steps: it reads anything you type (WER under 1%) ~200k steps: the voice stops sounding synthetic On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power. Some things we'd love to see people try: - Train it in your own language (a few hundred hours of speech gets you surprisingly far). - Add new features to Pocket TTS (Emotion tags? Make it sing?). - Beat us at our own game: make it faster and smaller. Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it! Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own. Get started:

kyutai

106,153 просмотров • 1 месяц назад

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."
2:40

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

TBPN

85,358 просмотров • 8 месяцев назад

Boom! I just opened what just may be the largest Apple HyperCard Stack Collection known: over 400,000 HyperCard Stacks for AI training! It was donated to me by a former Apple employee that saved every Stack they found, attending Apple User Groups across the country for over a decade and a half. Much of this content has never found its way in the Internet and some have mountains of data. Much of it is by folks that just wanted to make a place for unique data and ideas. It is a treasure trove! I now have an agent pipeline that will run the 5 disk DVD player until I load all the disks (over 100) for AI training. I suspect I will donate to online archives these Stacks at some point with permission form the estate. I can say I am blown away by this data set and know impart wisdom to YOUR AI. THANK YOU FOR FINDING ME HERE ON X AND TRUSTING ME WITH THIS LIFE CURATION AND COMMITMENT! More soon!
2:14

Boom! I just opened what just may be the largest Apple HyperCard Stack Collection known: over 400,000 HyperCard Stacks for AI training! It was donated to me by a former Apple employee that saved every Stack they found, attending Apple User Groups across the country for over a decade and a half. Much of this content has never found its way in the Internet and some have mountains of data. Much of it is by folks that just wanted to make a place for unique data and ideas. It is a treasure trove! I now have an agent pipeline that will run the 5 disk DVD player until I load all the disks (over 100) for AI training. I suspect I will donate to online archives these Stacks at some point with permission form the estate. I can say I am blown away by this data set and know impart wisdom to YOUR AI. THANK YOU FOR FINDING ME HERE ON X AND TRUSTING ME WITH THIS LIFE CURATION AND COMMITMENT! More soon!

Brian Roemmele

109,976 просмотров • 6 месяцев назад

Here is a live demo of our AI solution I've been building non-stop over the past 8 months Binary Defense. How it works: Our own model trained on our analysts behavior. Our analysts submit tickets as false positives/true positives with context which enriches our LLM to be smarter over time. Key Highlights: If its a binary - will automatically spin up an agent for reverse engineering it and using EMBER ML to understand behavior and intent of the binary. File formats: Supports a vast array of pretty much any filetype, including email attachments like SVG, LNK, etc. Can handle DLLs, ELF, EXEs, PDF, XLS, DOC, etc. Interrogates the full chain of all events irrespective of log sources. Can handle any format of logs and integrates into APIs of customers for additional agentic data looping for confidence ranking when needed. This is an example of the back-end UI, this is transparent to analysts and enriches the alarms automatically in our SOAR. In these examples there's three different types: 1. Regsvr32 + sct downloader + scrobj.dll code execution - checks reputation of domain, pulls in threat intel, looks at entire picture of the chain - downloads the file itself and inspects for code analysis. Determines if malicious as well as historically looking back if seen in customer before in past. 2. Powershell Obfuscation - uses a universal decoder to un-obfuscate powershell and look at the raw code. Can handle pretty much any obfuscation thrown at it (thanks Justin Elze). 3. Email with malicious SVG - checks tonality of email, are they creating urgency to take action (increases confidence) - disassembles SVG to understand malicious content - checks URL to determine if harvesting credentials, payload delivery, etc. Creates an entire kill chain analysis with full response and dissecting of the attack to the analyst in seconds. Has greatly sped up our ability to respond to incidents and allowing analysts to focus on the most important alarms through prioritization. Once cool thing I've worked heavily on is a synthetic data normalizer which when an analyst says "Yes this is bad with context" or "No this is a false positive" - our local model generates training data to be smarter in the future without using the actual customer data to train it. The customers actual data is immediately destroyed once training data off of the original alarm is generated and contains no customer-centric data at all. We also have three model tiers. Opt-In (collective model, again no customer data but every organization contributes to training). Opt-Out - does not train on any customer data for customers who opt-out. Private LLM - LLM created specifically for individual customer and trains only off of their data. Uses shared model collective for better confidence rankings. It will generate automated playbooks to run based on confidence rankings to take action on behalf of the customer. Still human driven on execution - has to approve playbook actions. This thing is cooking and so cool to see this work live and shut down attackers much faster! If confidence ranking is low - will automatically attempt to enrich data through customer environments for better confidence rankings. Additionally if the model isn't trained well on a certain technology, I have created something we call "Nexus" that will research new protocols, devices, SDKs, etc and generate training data automatically. Works well for zero-days for example, point to a tweet, or a research paper, and automatically generates training data to recognize this attack much faster. Have over 8000+ yara rule integrations that help with confidence boosting as well that is automatically incorporated into the analysis. Creating some amazing stuff at Binary Defense that isn't marketing fluff - actionable things that are making a huge difference in this industry. #BinaryDefense
0:32

Here is a live demo of our AI solution I've been building non-stop over the past 8 months Binary Defense. How it works: Our own model trained on our analysts behavior. Our analysts submit tickets as false positives/true positives with context which enriches our LLM to be smarter over time. Key Highlights: If its a binary - will automatically spin up an agent for reverse engineering it and using EMBER ML to understand behavior and intent of the binary. File formats: Supports a vast array of pretty much any filetype, including email attachments like SVG, LNK, etc. Can handle DLLs, ELF, EXEs, PDF, XLS, DOC, etc. Interrogates the full chain of all events irrespective of log sources. Can handle any format of logs and integrates into APIs of customers for additional agentic data looping for confidence ranking when needed. This is an example of the back-end UI, this is transparent to analysts and enriches the alarms automatically in our SOAR. In these examples there's three different types: 1. Regsvr32 + sct downloader + scrobj.dll code execution - checks reputation of domain, pulls in threat intel, looks at entire picture of the chain - downloads the file itself and inspects for code analysis. Determines if malicious as well as historically looking back if seen in customer before in past. 2. Powershell Obfuscation - uses a universal decoder to un-obfuscate powershell and look at the raw code. Can handle pretty much any obfuscation thrown at it (thanks Justin Elze). 3. Email with malicious SVG - checks tonality of email, are they creating urgency to take action (increases confidence) - disassembles SVG to understand malicious content - checks URL to determine if harvesting credentials, payload delivery, etc. Creates an entire kill chain analysis with full response and dissecting of the attack to the analyst in seconds. Has greatly sped up our ability to respond to incidents and allowing analysts to focus on the most important alarms through prioritization. Once cool thing I've worked heavily on is a synthetic data normalizer which when an analyst says "Yes this is bad with context" or "No this is a false positive" - our local model generates training data to be smarter in the future without using the actual customer data to train it. The customers actual data is immediately destroyed once training data off of the original alarm is generated and contains no customer-centric data at all. We also have three model tiers. Opt-In (collective model, again no customer data but every organization contributes to training). Opt-Out - does not train on any customer data for customers who opt-out. Private LLM - LLM created specifically for individual customer and trains only off of their data. Uses shared model collective for better confidence rankings. It will generate automated playbooks to run based on confidence rankings to take action on behalf of the customer. Still human driven on execution - has to approve playbook actions. This thing is cooking and so cool to see this work live and shut down attackers much faster! If confidence ranking is low - will automatically attempt to enrich data through customer environments for better confidence rankings. Additionally if the model isn't trained well on a certain technology, I have created something we call "Nexus" that will research new protocols, devices, SDKs, etc and generate training data automatically. Works well for zero-days for example, point to a tweet, or a research paper, and automatically generates training data to recognize this attack much faster. Have over 8000+ yara rule integrations that help with confidence boosting as well that is automatically incorporated into the analysis. Creating some amazing stuff at Binary Defense that isn't marketing fluff - actionable things that are making a huge difference in this industry. #BinaryDefense

Dave Kennedy

29,036 просмотров • 7 месяцев назад

Cognition President Russell Kaplan reveals the Cerebras chips they deployed ran their own models so fast at about 950 tokens per second that the team was confused by the test results, and it enabled them to ship a product experience that wouldn't have been possible otherwise. "One thing that was really fun working on SWE is we were the first kind of Western firm to deploy Cerebras at scale. You know, when we tested their chips, we were really confused because we were like, this is really good. You know, it operates a unique point on the Pareto curve of price and throughput." "And, you know, we were getting about 950 tokens per second on our own models, which was many x faster than what we could get on GPUs for the same size of model. And it was a little bit more expensive for us to serve. They were great partners, and they enabled us to ship a product experience that basically wouldn't have been possible otherwise." "And now Cerebras is more popular. You know, now, you know, OpenAI also uses Cerebras for Spark. I think more people are considering them. But we want to constantly be basically pushing the frontier of, well, what's the next most possible thing that wasn't possible previously? And yeah, doing our own RL and post-training helps with that."
0:52

Cognition President Russell Kaplan reveals the Cerebras chips they deployed ran their own models so fast at about 950 tokens per second that the team was confused by the test results, and it enabled them to ship a product experience that wouldn't have been possible otherwise. "One thing that was really fun working on SWE is we were the first kind of Western firm to deploy Cerebras at scale. You know, when we tested their chips, we were really confused because we were like, this is really good. You know, it operates a unique point on the Pareto curve of price and throughput." "And, you know, we were getting about 950 tokens per second on our own models, which was many x faster than what we could get on GPUs for the same size of model. And it was a little bit more expensive for us to serve. They were great partners, and they enabled us to ship a product experience that basically wouldn't have been possible otherwise." "And now Cerebras is more popular. You know, now, you know, OpenAI also uses Cerebras for Spark. I think more people are considering them. But we want to constantly be basically pushing the frontier of, well, what's the next most possible thing that wasn't possible previously? And yeah, doing our own RL and post-training helps with that."

Fireside Alpha

22,175 просмотров • 2 месяцев назад