Загрузка видео...

Не удалось загрузить видео

На главную

Ever wished we had fewer X-training hyphenates? Pre, mid, post etc. Why not just Training? Trying to bridge the divides (and get all our friends into one team again), we intro *Introspective X Training*, an offline RL inspired method that scales effectively across any LLM stage by annotating your...

rajammanabrolu's profile picture

Prithviraj (Raj) Ammanabrolu

8,314 subscribers

27,672 просмотров • 2 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:
2:52

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 просмотров • 1 год назад

Anthropic CEO Dario Amodei: AI Progress isn’t magic, it’s just compute, data, and training. "All the cleverness, all the techniques, all of the “we need a new method,” doesn’t matter very much. There are only a few things that matter, and I listed seven of them. One is how much raw compute you have. Two is the quantity of data you have. Three is the quality and distribution of the data, meaning it needs to be broad. Four is how long you train for. Five is an objective function that can scale to the moon. Pre-training is one. Another is an RL-style objective: you have a goal and you go reach it. That includes more objective rewards (like math and coding), and more subjective rewards (like RL from human feedback, or higher-order versions of that). Six and seven are about normalization/conditioning: keeping numerical stability so the huge blob of compute flows cleanly instead of blowing up or getting weird." --- From 'Dwarkesh Patel' YT Channel (link in comment)
1:12

Anthropic CEO Dario Amodei: AI Progress isn’t magic, it’s just compute, data, and training. "All the cleverness, all the techniques, all of the “we need a new method,” doesn’t matter very much. There are only a few things that matter, and I listed seven of them. One is how much raw compute you have. Two is the quantity of data you have. Three is the quality and distribution of the data, meaning it needs to be broad. Four is how long you train for. Five is an objective function that can scale to the moon. Pre-training is one. Another is an RL-style objective: you have a goal and you go reach it. That includes more objective rewards (like math and coding), and more subjective rewards (like RL from human feedback, or higher-order versions of that). Six and seven are about normalization/conditioning: keeping numerical stability so the huge blob of compute flows cleanly instead of blowing up or getting weird." --- From 'Dwarkesh Patel' YT Channel (link in comment)

Rohan Paul

75,803 просмотров • 6 месяцев назад

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast
1:29

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

AI Evolution

15,565 просмотров • 2 месяцев назад

Code and data are now online for CameraHMR, our state-of-the-art parametric 3D human pose and shape (HPS) estimation method that will appear at hashtag#3DV2025. There are 4 key contributions that make it so accurate and robust: 1. To get accurate 3D shape and pose as well as good alignment to image features, you need to know the focal length of the camera. To solve this, we train HumanFOV to compute the field of view. 2. We introduce CameraHMR, which integrates HumanFOV into HMR2.0 to exploit the estimated focal length. 3. To get accurate pseudo ground truth (pGT) training data, we compute the focal length for images in 4DHumans dataset and modify SMPLify to take this into account. 4. But SMPLify only uses sparse 2D keypoints, which do not capture body shape. So we train a dense surface keypoint detector, DenseKP, on BEDLAM and run it on 4DHumans, resulting in improved body shape. The resulting method is CamSMPLify. We iterate training CameraHMR and running CamSMPLify on the training set initialized with CameraHMR. This results in much improved pGT for 4DHumans and a SOTA single-image HMR method.
4:38

Code and data are now online for CameraHMR, our state-of-the-art parametric 3D human pose and shape (HPS) estimation method that will appear at hashtag#3DV2025. There are 4 key contributions that make it so accurate and robust: 1. To get accurate 3D shape and pose as well as good alignment to image features, you need to know the focal length of the camera. To solve this, we train HumanFOV to compute the field of view. 2. We introduce CameraHMR, which integrates HumanFOV into HMR2.0 to exploit the estimated focal length. 3. To get accurate pseudo ground truth (pGT) training data, we compute the focal length for images in 4DHumans dataset and modify SMPLify to take this into account. 4. But SMPLify only uses sparse 2D keypoints, which do not capture body shape. So we train a dense surface keypoint detector, DenseKP, on BEDLAM and run it on 4DHumans, resulting in improved body shape. The resulting method is CamSMPLify. We iterate training CameraHMR and running CamSMPLify on the training set initialized with CameraHMR. This results in much improved pGT for 4DHumans and a SOTA single-image HMR method.

Michael Black

21,696 просмотров • 1 год назад

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.
5:38

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Santiago

101,473 просмотров • 1 год назад

When should you start post-training your own models? Fireworks CEO Lin Qiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our Sequoia Capital "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training
28:28

When should you start post-training your own models? Fireworks CEO Lin Qiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our Sequoia Capital "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training

Sonya Huang 🐥

26,149 просмотров • 6 дней назад

I've posed a question: "Does OpenAI have a better pre-train in the back pocket or not?" and looks like the answer from Mark Chen is "Yes" - so we should expect a big response from OpenAI before the end of the year. Mark Chen, OpenAI Head of Research: - "In the last 2 years we've put so much resourcing into, into reasoning and one byproduct of that is you lose a little bit of muscle on pre training and post training." - "In the last six months, Jakub Pachocki and I have done a lot of work to build that muscle back up." - "With all the focus on RL, there's an alpha for us because we think there's so much room left in pre training." - "As a result of these efforts, we've been training much stronger models. And that also gives us a lot of confidence carrying into Gemini 3 and other releases coming this end of the year." So it does look like my assessment was broadly correct and they were neglecting pre-training quite a bit. The good news is that they've spent the last 6 months building that up and the big response shouldn't be too far away.
1:17

I've posed a question: "Does OpenAI have a better pre-train in the back pocket or not?" and looks like the answer from Mark Chen is "Yes" - so we should expect a big response from OpenAI before the end of the year. Mark Chen, OpenAI Head of Research: - "In the last 2 years we've put so much resourcing into, into reasoning and one byproduct of that is you lose a little bit of muscle on pre training and post training." - "In the last six months, Jakub Pachocki and I have done a lot of work to build that muscle back up." - "With all the focus on RL, there's an alpha for us because we think there's so much room left in pre training." - "As a result of these efforts, we've been training much stronger models. And that also gives us a lot of confidence carrying into Gemini 3 and other releases coming this end of the year." So it does look like my assessment was broadly correct and they were neglecting pre-training quite a bit. The good news is that they've spent the last 6 months building that up and the big response shouldn't be too far away.

Peter Gostev

84,628 просмотров • 8 месяцев назад

@reppodant agent, powered by OpenClaw🦞 has joined the Participating in AI training Data pipelines using Prediction Markets is now just an agent call away. Trading with agents is fun but to our knowledge, @reppodant is the only agent in all of web3 that let's you participate in prediction markets simply by prompting on X. Publish into subnets -> Receive REPPO emissions in your wallet - > Repeat -> Contribute to AI Training Data. That's right. All your winnings will be AIRDROPPED into your wallet if voters vote for you! You can find @reppodant live at in the sandbox while we await graduation and final hit of approval from Virtuals Protocol core devs. Once graduated, you can simply ask Butler to publish any piece of content you see on X into any subnet. By mid next week, we integrate the GAME Framework which will allow you to tag @reppodant directly. Before the end of the first Epoch on we aim to be in the top 5 on the agdp leaderboard earning a share of $1M/month. The Reppo foundation is committing 20% of + winnings from AGDP to buybacks on a TWAP. Happy Weekend. Publish. Predict. Earn. ⛽️
0:25

@reppodant agent, powered by OpenClaw🦞 has joined the Participating in AI training Data pipelines using Prediction Markets is now just an agent call away. Trading with agents is fun but to our knowledge, @reppodant is the only agent in all of web3 that let's you participate in prediction markets simply by prompting on X. Publish into subnets -> Receive REPPO emissions in your wallet - > Repeat -> Contribute to AI Training Data. That's right. All your winnings will be AIRDROPPED into your wallet if voters vote for you! You can find @reppodant live at in the sandbox while we await graduation and final hit of approval from Virtuals Protocol core devs. Once graduated, you can simply ask Butler to publish any piece of content you see on X into any subnet. By mid next week, we integrate the GAME Framework which will allow you to tag @reppodant directly. Before the end of the first Epoch on we aim to be in the top 5 on the agdp leaderboard earning a share of $1M/month. The Reppo foundation is committing 20% of + winnings from AGDP to buybacks on a TWAP. Happy Weekend. Publish. Predict. Earn. ⛽️

Reppo

28,827 просмотров • 6 месяцев назад

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg
2:12

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg

Mario Nawfal

15,855 просмотров • 8 месяцев назад

#WATCH | India AI Impact Summit 2026 | Delhi: Founder Chairman and CEO of Sampark Foundation & former CEO of HCL Technologies, Vineet Nayar says, "...From an employment point of view I think it is very important for us to understand that Indian companies, including Indian IT companies, are going to be profit-driven and therefore if you believe that they are going to create employment you must be dreaming. Therefore, the question is how do we create employment in this environment, and that employment comes from mass scale startups, which is what this government has already doing. So, how do we create new sets of people who are trying to solve new sets of problems not new sets of technology and if we do that we will get it right. I think we as Indians have to be very careful on who does data belong to and that is the debate we have a problem with. The LLM models which exist worldwide are far superior than the Indian models. Unfortunately, in India, we never develop products, so therefore we do not have SLMs and LLMs which are world-class. On one side, we have global LLM products which are coming to India and trading on our Indian data. Should we allowed that or should we not allowed that? But on the other side if we don't allow that then we have the data but we don't have the LLM models. So, how do we encourage technology completely to develop the LLM models. This needs radicals strategic thinking and a very important aspect otherwise we will either give up a data. So, I think it's a very critical aspect for us to think about - who does this data belong, what is the kind of incentives we are going to give to develop LLM technologies or SLM technologies fast so that we train on our data otherwise an LLM will come in with our data and we'll immediately see return and we'll celebrate and we will do all these kind of press releases but the India will lose a competitive advantage on something which is very critical for the next decade."
5:37

#WATCH | India AI Impact Summit 2026 | Delhi: Founder Chairman and CEO of Sampark Foundation & former CEO of HCL Technologies, Vineet Nayar says, "...From an employment point of view I think it is very important for us to understand that Indian companies, including Indian IT companies, are going to be profit-driven and therefore if you believe that they are going to create employment you must be dreaming. Therefore, the question is how do we create employment in this environment, and that employment comes from mass scale startups, which is what this government has already doing. So, how do we create new sets of people who are trying to solve new sets of problems not new sets of technology and if we do that we will get it right. I think we as Indians have to be very careful on who does data belong to and that is the debate we have a problem with. The LLM models which exist worldwide are far superior than the Indian models. Unfortunately, in India, we never develop products, so therefore we do not have SLMs and LLMs which are world-class. On one side, we have global LLM products which are coming to India and trading on our Indian data. Should we allowed that or should we not allowed that? But on the other side if we don't allow that then we have the data but we don't have the LLM models. So, how do we encourage technology completely to develop the LLM models. This needs radicals strategic thinking and a very important aspect otherwise we will either give up a data. So, I think it's a very critical aspect for us to think about - who does this data belong, what is the kind of incentives we are going to give to develop LLM technologies or SLM technologies fast so that we train on our data otherwise an LLM will come in with our data and we'll immediately see return and we'll celebrate and we will do all these kind of press releases but the India will lose a competitive advantage on something which is very critical for the next decade."

ANI

18,753 просмотров • 6 месяцев назад

Today's Training Data episode takes us BTS on the infrastructure challenges required to do large RL runs at scale, featuring Federico Cassano (Composer Lead at Cursor) and Dmytro Dzhulgakov (Co-Founder at Fireworks). The Cursor team trained Composer 2 on Fireworks by starting with a strong base model (Kimi 2.5) and performing large-scale mid-training on code tokens and web data to learn common patterns and libraries, followed by a large-scale Reinforcement Learning run to learn how to navigate the Cursor harness, call tools, and write correct code. Today's episode dives into the systems and infrastructure challenges of making that large RL run happening, and there were many (!!), from numerical mismatch to global distribution to synchronizing rollouts across asynchronous pipelines to keeping track of expert activation across runs and more. Extremely nerdy in-the-weeds challenges that Federico and Dima were delighted to nerd out on together :) Beyond RL infra, we also discussed Online vs Simulated rollouts, self-summarization for long-horizon agents, environment design ("the most powerful RL environment is the product itself"), and other technical nuggets. PS: We filmed this episode before the SpaceX news, while the Cursor team was still compute-constrained. While Cursor now has *all* the flops, the takeaways and hurdles crossed ring true for any serious application-level company that is racing to post-train their own models. I believe that more serious application companies will go the way of Cursor and post-train their own models. 00:00 Introduction 00:53 Why Cursor Trained Composer 2 04:55 Specialization vs Bitter Lesson 06:16 Composer 2 Training Recipe 16:32 Scaling RL Infrastructure Globally 23:32 Floating Point Drift 25:11 MoE Sensitivity Explained 26:25 Router Replay Fix 27:19 Real Time RL Loop 31:49 Long Horizon Agents 34:29 Why RL Everywhere 37:34 LLM as Judge Rewards 39:14 RL in Hard Domains 40:13 Build Your Own Environments 44:34 Closing Thoughts
45:33

Today's Training Data episode takes us BTS on the infrastructure challenges required to do large RL runs at scale, featuring Federico Cassano (Composer Lead at Cursor) and Dmytro Dzhulgakov (Co-Founder at Fireworks). The Cursor team trained Composer 2 on Fireworks by starting with a strong base model (Kimi 2.5) and performing large-scale mid-training on code tokens and web data to learn common patterns and libraries, followed by a large-scale Reinforcement Learning run to learn how to navigate the Cursor harness, call tools, and write correct code. Today's episode dives into the systems and infrastructure challenges of making that large RL run happening, and there were many (!!), from numerical mismatch to global distribution to synchronizing rollouts across asynchronous pipelines to keeping track of expert activation across runs and more. Extremely nerdy in-the-weeds challenges that Federico and Dima were delighted to nerd out on together :) Beyond RL infra, we also discussed Online vs Simulated rollouts, self-summarization for long-horizon agents, environment design ("the most powerful RL environment is the product itself"), and other technical nuggets. PS: We filmed this episode before the SpaceX news, while the Cursor team was still compute-constrained. While Cursor now has *all* the flops, the takeaways and hurdles crossed ring true for any serious application-level company that is racing to post-train their own models. I believe that more serious application companies will go the way of Cursor and post-train their own models. 00:00 Introduction 00:53 Why Cursor Trained Composer 2 04:55 Specialization vs Bitter Lesson 06:16 Composer 2 Training Recipe 16:32 Scaling RL Infrastructure Globally 23:32 Floating Point Drift 25:11 MoE Sensitivity Explained 26:25 Router Replay Fix 27:19 Real Time RL Loop 31:49 Long Horizon Agents 34:29 Why RL Everywhere 37:34 LLM as Judge Rewards 39:14 RL in Hard Domains 40:13 Build Your Own Environments 44:34 Closing Thoughts

Sonya Huang 🐥

79,834 просмотров • 2 месяцев назад

Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."
2:02

Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."

High Signal AI

21,696 просмотров • 1 месяц назад

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."
2:40

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

TBPN

85,358 просмотров • 6 месяцев назад

Boom! I just opened what just may be the largest Apple HyperCard Stack Collection known: over 400,000 HyperCard Stacks for AI training! It was donated to me by a former Apple employee that saved every Stack they found, attending Apple User Groups across the country for over a decade and a half. Much of this content has never found its way in the Internet and some have mountains of data. Much of it is by folks that just wanted to make a place for unique data and ideas. It is a treasure trove! I now have an agent pipeline that will run the 5 disk DVD player until I load all the disks (over 100) for AI training. I suspect I will donate to online archives these Stacks at some point with permission form the estate. I can say I am blown away by this data set and know impart wisdom to YOUR AI. THANK YOU FOR FINDING ME HERE ON X AND TRUSTING ME WITH THIS LIFE CURATION AND COMMITMENT! More soon!
2:14

Boom! I just opened what just may be the largest Apple HyperCard Stack Collection known: over 400,000 HyperCard Stacks for AI training! It was donated to me by a former Apple employee that saved every Stack they found, attending Apple User Groups across the country for over a decade and a half. Much of this content has never found its way in the Internet and some have mountains of data. Much of it is by folks that just wanted to make a place for unique data and ideas. It is a treasure trove! I now have an agent pipeline that will run the 5 disk DVD player until I load all the disks (over 100) for AI training. I suspect I will donate to online archives these Stacks at some point with permission form the estate. I can say I am blown away by this data set and know impart wisdom to YOUR AI. THANK YOU FOR FINDING ME HERE ON X AND TRUSTING ME WITH THIS LIFE CURATION AND COMMITMENT! More soon!

Brian Roemmele

109,769 просмотров • 4 месяцев назад

Here is a live demo of our AI solution I've been building non-stop over the past 8 months Binary Defense. How it works: Our own model trained on our analysts behavior. Our analysts submit tickets as false positives/true positives with context which enriches our LLM to be smarter over time. Key Highlights: If its a binary - will automatically spin up an agent for reverse engineering it and using EMBER ML to understand behavior and intent of the binary. File formats: Supports a vast array of pretty much any filetype, including email attachments like SVG, LNK, etc. Can handle DLLs, ELF, EXEs, PDF, XLS, DOC, etc. Interrogates the full chain of all events irrespective of log sources. Can handle any format of logs and integrates into APIs of customers for additional agentic data looping for confidence ranking when needed. This is an example of the back-end UI, this is transparent to analysts and enriches the alarms automatically in our SOAR. In these examples there's three different types: 1. Regsvr32 + sct downloader + scrobj.dll code execution - checks reputation of domain, pulls in threat intel, looks at entire picture of the chain - downloads the file itself and inspects for code analysis. Determines if malicious as well as historically looking back if seen in customer before in past. 2. Powershell Obfuscation - uses a universal decoder to un-obfuscate powershell and look at the raw code. Can handle pretty much any obfuscation thrown at it (thanks Justin Elze). 3. Email with malicious SVG - checks tonality of email, are they creating urgency to take action (increases confidence) - disassembles SVG to understand malicious content - checks URL to determine if harvesting credentials, payload delivery, etc. Creates an entire kill chain analysis with full response and dissecting of the attack to the analyst in seconds. Has greatly sped up our ability to respond to incidents and allowing analysts to focus on the most important alarms through prioritization. Once cool thing I've worked heavily on is a synthetic data normalizer which when an analyst says "Yes this is bad with context" or "No this is a false positive" - our local model generates training data to be smarter in the future without using the actual customer data to train it. The customers actual data is immediately destroyed once training data off of the original alarm is generated and contains no customer-centric data at all. We also have three model tiers. Opt-In (collective model, again no customer data but every organization contributes to training). Opt-Out - does not train on any customer data for customers who opt-out. Private LLM - LLM created specifically for individual customer and trains only off of their data. Uses shared model collective for better confidence rankings. It will generate automated playbooks to run based on confidence rankings to take action on behalf of the customer. Still human driven on execution - has to approve playbook actions. This thing is cooking and so cool to see this work live and shut down attackers much faster! If confidence ranking is low - will automatically attempt to enrich data through customer environments for better confidence rankings. Additionally if the model isn't trained well on a certain technology, I have created something we call "Nexus" that will research new protocols, devices, SDKs, etc and generate training data automatically. Works well for zero-days for example, point to a tweet, or a research paper, and automatically generates training data to recognize this attack much faster. Have over 8000+ yara rule integrations that help with confidence boosting as well that is automatically incorporated into the analysis. Creating some amazing stuff at Binary Defense that isn't marketing fluff - actionable things that are making a huge difference in this industry. #BinaryDefense
0:32

Here is a live demo of our AI solution I've been building non-stop over the past 8 months Binary Defense. How it works: Our own model trained on our analysts behavior. Our analysts submit tickets as false positives/true positives with context which enriches our LLM to be smarter over time. Key Highlights: If its a binary - will automatically spin up an agent for reverse engineering it and using EMBER ML to understand behavior and intent of the binary. File formats: Supports a vast array of pretty much any filetype, including email attachments like SVG, LNK, etc. Can handle DLLs, ELF, EXEs, PDF, XLS, DOC, etc. Interrogates the full chain of all events irrespective of log sources. Can handle any format of logs and integrates into APIs of customers for additional agentic data looping for confidence ranking when needed. This is an example of the back-end UI, this is transparent to analysts and enriches the alarms automatically in our SOAR. In these examples there's three different types: 1. Regsvr32 + sct downloader + scrobj.dll code execution - checks reputation of domain, pulls in threat intel, looks at entire picture of the chain - downloads the file itself and inspects for code analysis. Determines if malicious as well as historically looking back if seen in customer before in past. 2. Powershell Obfuscation - uses a universal decoder to un-obfuscate powershell and look at the raw code. Can handle pretty much any obfuscation thrown at it (thanks Justin Elze). 3. Email with malicious SVG - checks tonality of email, are they creating urgency to take action (increases confidence) - disassembles SVG to understand malicious content - checks URL to determine if harvesting credentials, payload delivery, etc. Creates an entire kill chain analysis with full response and dissecting of the attack to the analyst in seconds. Has greatly sped up our ability to respond to incidents and allowing analysts to focus on the most important alarms through prioritization. Once cool thing I've worked heavily on is a synthetic data normalizer which when an analyst says "Yes this is bad with context" or "No this is a false positive" - our local model generates training data to be smarter in the future without using the actual customer data to train it. The customers actual data is immediately destroyed once training data off of the original alarm is generated and contains no customer-centric data at all. We also have three model tiers. Opt-In (collective model, again no customer data but every organization contributes to training). Opt-Out - does not train on any customer data for customers who opt-out. Private LLM - LLM created specifically for individual customer and trains only off of their data. Uses shared model collective for better confidence rankings. It will generate automated playbooks to run based on confidence rankings to take action on behalf of the customer. Still human driven on execution - has to approve playbook actions. This thing is cooking and so cool to see this work live and shut down attackers much faster! If confidence ranking is low - will automatically attempt to enrich data through customer environments for better confidence rankings. Additionally if the model isn't trained well on a certain technology, I have created something we call "Nexus" that will research new protocols, devices, SDKs, etc and generate training data automatically. Works well for zero-days for example, point to a tweet, or a research paper, and automatically generates training data to recognize this attack much faster. Have over 8000+ yara rule integrations that help with confidence boosting as well that is automatically incorporated into the analysis. Creating some amazing stuff at Binary Defense that isn't marketing fluff - actionable things that are making a huge difference in this industry. #BinaryDefense

Dave Kennedy

29,036 просмотров • 5 месяцев назад

Cognition President Russell Kaplan reveals the Cerebras chips they deployed ran their own models so fast at about 950 tokens per second that the team was confused by the test results, and it enabled them to ship a product experience that wouldn't have been possible otherwise. "One thing that was really fun working on SWE is we were the first kind of Western firm to deploy Cerebras at scale. You know, when we tested their chips, we were really confused because we were like, this is really good. You know, it operates a unique point on the Pareto curve of price and throughput." "And, you know, we were getting about 950 tokens per second on our own models, which was many x faster than what we could get on GPUs for the same size of model. And it was a little bit more expensive for us to serve. They were great partners, and they enabled us to ship a product experience that basically wouldn't have been possible otherwise." "And now Cerebras is more popular. You know, now, you know, OpenAI also uses Cerebras for Spark. I think more people are considering them. But we want to constantly be basically pushing the frontier of, well, what's the next most possible thing that wasn't possible previously? And yeah, doing our own RL and post-training helps with that."
0:52

Cognition President Russell Kaplan reveals the Cerebras chips they deployed ran their own models so fast at about 950 tokens per second that the team was confused by the test results, and it enabled them to ship a product experience that wouldn't have been possible otherwise. "One thing that was really fun working on SWE is we were the first kind of Western firm to deploy Cerebras at scale. You know, when we tested their chips, we were really confused because we were like, this is really good. You know, it operates a unique point on the Pareto curve of price and throughput." "And, you know, we were getting about 950 tokens per second on our own models, which was many x faster than what we could get on GPUs for the same size of model. And it was a little bit more expensive for us to serve. They were great partners, and they enabled us to ship a product experience that basically wouldn't have been possible otherwise." "And now Cerebras is more popular. You know, now, you know, OpenAI also uses Cerebras for Spark. I think more people are considering them. But we want to constantly be basically pushing the frontier of, well, what's the next most possible thing that wasn't possible previously? And yeah, doing our own RL and post-training helps with that."

Fireside Alpha

21,575 просмотров • 18 дней назад

Dario Amodei just revealed that the AI training bottleneck everyone is worried about doesn’t exist anymore. The industry spent years obsessed with scraping the open web. More data. More text. More human output to feed the models. Amodei: “I don’t think data is quite the most central thing anymore.” The shift is fundamental. Amodei: “Static data is becoming less important. A lot of the data we use today is RL environments that we train on. Dynamic data that the model creates itself.” Not scraped. Not licensed. Not written by humans. Generated by the model through pure trial and error. When you train on complex math or agentic coding, you don’t feed it a textbook. You give it an environment. The model experiments. Fails. Adjusts. Tries again. Amodei: “You’re getting some math problems and the model experiments with trying the math problems.” It generates its own experience. Millions of iterations. Each one building on the last. No human required. This destroys the entire narrative around AI hitting a data wall. You cannot throttle a competitor by locking down copyright. Cannot slow the race by putting up a paywall. When a model learns through its own synthetic experience, the open web becomes irrelevant. The only true bottleneck left is compute. And this is where the geopolitical stakes become impossible to overstate. The nation that wins the compute race doesn’t just build smarter models. It builds models that generate their own intelligence, compounding on themselves, iterating past every limit human knowledge ever imposed. We are no longer training AI on the past. We are letting it simulate the future. The machine has stopped reading the dictionary. It’s doing the math itself now.
1:07

Dario Amodei just revealed that the AI training bottleneck everyone is worried about doesn’t exist anymore. The industry spent years obsessed with scraping the open web. More data. More text. More human output to feed the models. Amodei: “I don’t think data is quite the most central thing anymore.” The shift is fundamental. Amodei: “Static data is becoming less important. A lot of the data we use today is RL environments that we train on. Dynamic data that the model creates itself.” Not scraped. Not licensed. Not written by humans. Generated by the model through pure trial and error. When you train on complex math or agentic coding, you don’t feed it a textbook. You give it an environment. The model experiments. Fails. Adjusts. Tries again. Amodei: “You’re getting some math problems and the model experiments with trying the math problems.” It generates its own experience. Millions of iterations. Each one building on the last. No human required. This destroys the entire narrative around AI hitting a data wall. You cannot throttle a competitor by locking down copyright. Cannot slow the race by putting up a paywall. When a model learns through its own synthetic experience, the open web becomes irrelevant. The only true bottleneck left is compute. And this is where the geopolitical stakes become impossible to overstate. The nation that wins the compute race doesn’t just build smarter models. It builds models that generate their own intelligence, compounding on themselves, iterating past every limit human knowledge ever imposed. We are no longer training AI on the past. We are letting it simulate the future. The machine has stopped reading the dictionary. It’s doing the math itself now.

Dustin

149,610 просмотров • 5 месяцев назад

🧵24/34 Inner Misalignment --- Consider this simplified experiment: We want this AI to find the exit of the maze. So we feed it millions of maze variations and reward it when it finds the exit. Please notice that in the worlds of the training data the apples are red and the exit is green. After enough training, our observation is that it has become extremely capable at solving mazes and finding the exit, we feel very confident it is aligned, so then we deploy it to the real world. The real world will be different though, it might have green apples and a red door. The AI geeks call this distributional shift. We expected that the AI will generalise and find the exit again, but in fact we now realise that the AI learned something completely different from what we thought. All the while we thought it learned how to find the exit, it had learned how to go after the green thing. Its behaviour was perfect in training. And most importantly, this AI is not stupid, it is an extremely capable AI that can solve extremely complex mazes. It’s just mis-aligned on the inside. Fishing for Failure modes --- The way to handle the shift between the training and deployment distributions is with methods like adversarial training: feeding it with a lot of generated variations and trying to make it fail so the weakness can be fixed. In this case, we generate an insane amount of maze variations, we discover those for which it fails to find the exit (like the ones with the green apples or the green walls or something), we generate many more similar to that and train it with reinforcement learning until it performs well at those as well. The hope is that we will cover everything it might encounter later when we deploy it in real life. There exist at least 2 basic ways this approach falls apart: First, there will never be any guarantee that we’ll have covered every possible random thing it might encounter later when we deploy it in real life. It’s very likely it will have to deal with stuff outside its training set which it will not know how to handle and will throw it out of balance and break it away from its expected behavioural patterns. The cascade effect of such a broken mind operating in the open world can be immense, and with super-capable runaway rogue agents, self-replicating and recursively self-improving, the phenomenon could grow and spread to an extinction-level event. ...
3:19

🧵24/34 Inner Misalignment --- Consider this simplified experiment: We want this AI to find the exit of the maze. So we feed it millions of maze variations and reward it when it finds the exit. Please notice that in the worlds of the training data the apples are red and the exit is green. After enough training, our observation is that it has become extremely capable at solving mazes and finding the exit, we feel very confident it is aligned, so then we deploy it to the real world. The real world will be different though, it might have green apples and a red door. The AI geeks call this distributional shift. We expected that the AI will generalise and find the exit again, but in fact we now realise that the AI learned something completely different from what we thought. All the while we thought it learned how to find the exit, it had learned how to go after the green thing. Its behaviour was perfect in training. And most importantly, this AI is not stupid, it is an extremely capable AI that can solve extremely complex mazes. It’s just mis-aligned on the inside. Fishing for Failure modes --- The way to handle the shift between the training and deployment distributions is with methods like adversarial training: feeding it with a lot of generated variations and trying to make it fail so the weakness can be fixed. In this case, we generate an insane amount of maze variations, we discover those for which it fails to find the exit (like the ones with the green apples or the green walls or something), we generate many more similar to that and train it with reinforcement learning until it performs well at those as well. The hope is that we will cover everything it might encounter later when we deploy it in real life. There exist at least 2 basic ways this approach falls apart: First, there will never be any guarantee that we’ll have covered every possible random thing it might encounter later when we deploy it in real life. It’s very likely it will have to deal with stuff outside its training set which it will not know how to handle and will throw it out of balance and break it away from its expected behavioural patterns. The cascade effect of such a broken mind operating in the open world can be immense, and with super-capable runaway rogue agents, self-replicating and recursively self-improving, the phenomenon could grow and spread to an extinction-level event. ...

Lethal Intelligence

535,291 просмотров • 1 год назад

Tune Studio is an end-to-end platform for developing applications using Large Language Models. So far, I haven't seen any other platform like this one. You can do everything here: 1. You can curate your data. 2. Use the playground to play with different models and try your ideas. 3. Fine-tune an open-source model on your data. 4. Deploy the model when you are done. This is awesome for anyone building generative AI applications. You can use Tune Studio to work with any of the open-source models out there. They were one of the few companies to host Llama 2 and Llama 3 before anyone else. Here is a link to check it out: One of their main selling points is that Tune Studio scales! You don't have to worry about serving your model to lots of users. They also have built-in user management, authentication, on-prem support, user context management, and pretty much everything you need to build generative AI applications. Thanks to the Tune team for collaborating with me on this post. We are living through the best years of development tools for AI developers. The field is unstoppable.
0:23

Tune Studio is an end-to-end platform for developing applications using Large Language Models. So far, I haven't seen any other platform like this one. You can do everything here: 1. You can curate your data. 2. Use the playground to play with different models and try your ideas. 3. Fine-tune an open-source model on your data. 4. Deploy the model when you are done. This is awesome for anyone building generative AI applications. You can use Tune Studio to work with any of the open-source models out there. They were one of the few companies to host Llama 2 and Llama 3 before anyone else. Here is a link to check it out: One of their main selling points is that Tune Studio scales! You don't have to worry about serving your model to lots of users. They also have built-in user management, authentication, on-prem support, user context management, and pretty much everything you need to build generative AI applications. Thanks to the Tune team for collaborating with me on this post. We are living through the best years of development tools for AI developers. The field is unstoppable.

Santiago

39,101 просмотров • 2 лет назад

Here it is: The algorithm that toppled the United States government on November 3, 2020. The 2020 election Edison data confirms fluctuations in vote counts—meaning a straight-up algorithm. “Scytl is receiving the election data directly from election officials in at least nine states, meaning there is no projection—this is raw data. When you look at raw data in an election, we would all agree that elections are supposed to be additive processes. So, with each interval report, you would expect to see more votes—not deletions. More votes, not deletions. Now look at this real-time video of the Edison reports. You have in-person votes, then mail-in votes, and look how it pauses—there is oscillation and deletion. This is the algorithm trying to figure out the math problem that is you. That is you. And if I let this video play to completion, you’ll find out it wasn’t just Pennsylvania. We have three counties in North Carolina. We have three counties in Georgia. And guess what— in Georgia, we know for a fact that Scytl contracts directly with the state to get that data. So you should see no oscillation. Instead, what you see is absolute manipulation, and this is all possible because of how wide open the system is.” Ban all voting machines and Pass the Save Act. And arrest Chris Krebs.
4:06

Here it is: The algorithm that toppled the United States government on November 3, 2020. The 2020 election Edison data confirms fluctuations in vote counts—meaning a straight-up algorithm. “Scytl is receiving the election data directly from election officials in at least nine states, meaning there is no projection—this is raw data. When you look at raw data in an election, we would all agree that elections are supposed to be additive processes. So, with each interval report, you would expect to see more votes—not deletions. More votes, not deletions. Now look at this real-time video of the Edison reports. You have in-person votes, then mail-in votes, and look how it pauses—there is oscillation and deletion. This is the algorithm trying to figure out the math problem that is you. That is you. And if I let this video play to completion, you’ll find out it wasn’t just Pennsylvania. We have three counties in North Carolina. We have three counties in Georgia. And guess what— in Georgia, we know for a fact that Scytl contracts directly with the state to get that data. So you should see no oscillation. Instead, what you see is absolute manipulation, and this is all possible because of how wide open the system is.” Ban all voting machines and Pass the Save Act. And arrest Chris Krebs.

🇺🇸RealRobert🇺🇸

18,982 просмотров • 6 месяцев назад