Loading video...

Video Failed to Load

Go Home

SITUATION EXPLAINED: Richard Sutton, the father of reinforcement learning, just launched a new lab. The goal: a trillion-parameter agent that learns in real time on 20 watts. • Sutton wrote The Bitter Lesson in 2019, its core claim: general methods that leverage computation beat hand-crafted approaches, by a large...

MTSlive's profile picture

MTS

281,707 subscribers

83,194 views • 6 days ago •via X (Twitter)

Science & TechnologyEducation
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

THE MAN WHO WROTE THE ONE ESSAY EVERY AI LAB TREATS AS SCRIPTURE NOW SAYS THE THING THEY BUILT WITH IT IS A DEAD END. THE FATHER OF THE "JUST SCALE IT" GOSPEL IS TELLING YOU THAT STACKING MORE LLMs LEADS NOWHERE 67 minutes with Richard Sutton -- father of reinforcement learning and winner of the 2024 Turing Award, the highest honor in computer science. -> His argument: an LLM only ever learns from human text. It imitates what people already wrote, but it never learns a single thing from its own experience of the world. A squirrel, a baby, even a rat learns on the job -- try something, see what happens, update. An LLM finishes training and then it is frozen. It cannot actually learn from you. So no matter how much compute you pour in, you are scaling a better and better imitator, not something that understands. The ceiling is baked in. The brutal part: everyone quotes his "Bitter Lesson" to justify the scaling race. He says they misread him -- real intelligence comes from learning through experience, and that piece is still completely missing. You assumed bigger models were the road to real AI. This is the father of the field telling you it is the wrong one. Save this. It is the bear case straight from the source ↓
1:06:59

THE MAN WHO WROTE THE ONE ESSAY EVERY AI LAB TREATS AS SCRIPTURE NOW SAYS THE THING THEY BUILT WITH IT IS A DEAD END. THE FATHER OF THE "JUST SCALE IT" GOSPEL IS TELLING YOU THAT STACKING MORE LLMs LEADS NOWHERE 67 minutes with Richard Sutton -- father of reinforcement learning and winner of the 2024 Turing Award, the highest honor in computer science. -> His argument: an LLM only ever learns from human text. It imitates what people already wrote, but it never learns a single thing from its own experience of the world. A squirrel, a baby, even a rat learns on the job -- try something, see what happens, update. An LLM finishes training and then it is frozen. It cannot actually learn from you. So no matter how much compute you pour in, you are scaling a better and better imitator, not something that understands. The ceiling is baked in. The brutal part: everyone quotes his "Bitter Lesson" to justify the scaling race. He says they misread him -- real intelligence comes from learning through experience, and that piece is still completely missing. You assumed bigger models were the road to real AI. This is the father of the field telling you it is the wrong one. Save this. It is the bear case straight from the source ↓

slash1s

12,623 views • 12 days ago

🚨 Chamath explains Grok 4's breakthrough and how Elon leapfrogged the competition in AI: Elon Musk and the team at xAI understood "The Bitter Lesson" by Richard Sutton Chamath Palihapitiya: "(The Bitter Lesson) basically says in a nutshell, that you're always better off, when you're trying to solve an AI problem, taking a general learning approach that can scale with computation. Because it ultimately proves to be the most effective." "The alternative would be something that's much more human labored and human involved that requires human knowledge." "So the first method, what it essentially allows you to do is view any problem as an endless, scalable search or learning task." "As it's turned out, whether it's chess, or Go, or speech recognition, or computer vision, whenever there was two competing approaches, one that used general computation and one that used human knowledge, the general computation problem always won." " And so it creates this bitter lesson for humans that want to think that we are at the center of all of this critical learning and all of these leaps, and what these results show is a general computational approach that doesn't require as much human labeling can actually get to the answer and better answers faster."
3:57

🚨 Chamath explains Grok 4's breakthrough and how Elon leapfrogged the competition in AI: Elon Musk and the team at xAI understood "The Bitter Lesson" by Richard Sutton Chamath Palihapitiya: "(The Bitter Lesson) basically says in a nutshell, that you're always better off, when you're trying to solve an AI problem, taking a general learning approach that can scale with computation. Because it ultimately proves to be the most effective." "The alternative would be something that's much more human labored and human involved that requires human knowledge." "So the first method, what it essentially allows you to do is view any problem as an endless, scalable search or learning task." "As it's turned out, whether it's chess, or Go, or speech recognition, or computer vision, whenever there was two competing approaches, one that used general computation and one that used human knowledge, the general computation problem always won." " And so it creates this bitter lesson for humans that want to think that we are at the center of all of this critical learning and all of these leaps, and what these results show is a general computational approach that doesn't require as much human labeling can actually get to the answer and better answers faster."

The All-In Podcast

597,321 views • 1 year ago

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.
1:41

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Dwarkesh Patel

54,245 views • 1 month ago

What if you could train AI agents on a laptop as easily as on a GPU cluster? Researchers from UIUC's U Lab, led by Prof. Jiaxuan You, just open-sourced OpenTinker. It's a new "Reinforcement-Learning-as-a-Service" (RLaaS) system that decouples the complex training pipeline into simple, distributed services with friendly APIs. The result? It breaks down the major engineering barriers to RL, outperforming traditional frameworks in accessibility and ease of deployment, finally making agent training viable for more developers and teams. Project: Code: U Lab: Our report: 📬 #PapersAccepted by Jiqizhixin
0:42

What if you could train AI agents on a laptop as easily as on a GPU cluster? Researchers from UIUC's U Lab, led by Prof. Jiaxuan You, just open-sourced OpenTinker. It's a new "Reinforcement-Learning-as-a-Service" (RLaaS) system that decouples the complex training pipeline into simple, distributed services with friendly APIs. The result? It breaks down the major engineering barriers to RL, outperforming traditional frameworks in accessibility and ease of deployment, finally making agent training viable for more developers and teams. Project: Code: U Lab: Our report: 📬 #PapersAccepted by Jiqizhixin

机器之心 JIQIZHIXIN

15,893 views • 6 months ago

.Richard Sutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled. My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning. And if we have continual learning, we don't need a special training phase - the agent just learns on-the-fly - like all humans, and indeed, like all animals. This new paradigm will render our current approach with LLMs obsolete. I did my best to represent the view that LLMs will function as the foundation on which this experiential learning can happen. Some sparks flew. 0:00:00 – Are LLMs a dead-end? 0:13:51 – Do humans do imitation learning? 0:23:57 – The Era of Experience 0:34:25 – Current architectures generalize poorly out of distribution 0:42:17 – Surprises in the AI field 0:47:28 – Will The Bitter Lesson still apply after AGI? 0:54:35 – Succession to AI
1:07:08

.Richard Sutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled. My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning. And if we have continual learning, we don't need a special training phase - the agent just learns on-the-fly - like all humans, and indeed, like all animals. This new paradigm will render our current approach with LLMs obsolete. I did my best to represent the view that LLMs will function as the foundation on which this experiential learning can happen. Some sparks flew. 0:00:00 – Are LLMs a dead-end? 0:13:51 – Do humans do imitation learning? 0:23:57 – The Era of Experience 0:34:25 – Current architectures generalize poorly out of distribution 0:42:17 – Surprises in the AI field 0:47:28 – Will The Bitter Lesson still apply after AGI? 0:54:35 – Succession to AI

Dwarkesh Patel

3,081,359 views • 9 months ago

Today is the beginning of our moonshot to solve embodied AGI in the physical world. I’m so excited to announce Project GR00T, our new initiative to create a general-purpose foundation model for humanoid robot learning. The GR00T model will enable a robot to understand multimodal instructions, such as language, video, and demonstration, and perform a variety of useful tasks. We are collaborating with many leading humanoid companies around the world, so that GR00T may transfer across embodiments and help the ecosystem thrive. GR00T is born on NVIDIA’s deep technology stack. We simulate in Isaac Lab (new app on Omniverse Isaac Sim for humanoid learning), train on OSMO (new compute orchestration system to scale up models), and deploy to Jetson Thor (new edge GPU chip designed to power GR00T). Announced in Jensen's keynote, Project GR00T is a cornerstone for the “Foundation Agent” roadmap of the newly founded GEAR Lab. At GEAR, we are building generally capable agents that learn to act skillfully in many worlds, virtual and real. See if you can spot "GEAR" in the video ;) Join us on the journey to land on the moon.
2:41

Today is the beginning of our moonshot to solve embodied AGI in the physical world. I’m so excited to announce Project GR00T, our new initiative to create a general-purpose foundation model for humanoid robot learning. The GR00T model will enable a robot to understand multimodal instructions, such as language, video, and demonstration, and perform a variety of useful tasks. We are collaborating with many leading humanoid companies around the world, so that GR00T may transfer across embodiments and help the ecosystem thrive. GR00T is born on NVIDIA’s deep technology stack. We simulate in Isaac Lab (new app on Omniverse Isaac Sim for humanoid learning), train on OSMO (new compute orchestration system to scale up models), and deploy to Jetson Thor (new edge GPU chip designed to power GR00T). Announced in Jensen's keynote, Project GR00T is a cornerstone for the “Foundation Agent” roadmap of the newly founded GEAR Lab. At GEAR, we are building generally capable agents that learn to act skillfully in many worlds, virtual and real. See if you can spot "GEAR" in the video ;) Join us on the journey to land on the moon.

Jim Fan

1,076,844 views • 2 years ago

The most interesting part for me is where Andrej Karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.” A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer. > “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.” So what do humans do instead? > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” > “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.” Why can’t we just add this training to LLMs today? > “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.” > “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.” > “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.” How do humans get around model collapse? > “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.” In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by Erik Hoel. I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization? > “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”
7:01

The most interesting part for me is where Andrej Karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.” A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer. > “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.” So what do humans do instead? > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” > “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.” Why can’t we just add this training to LLMs today? > “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.” > “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.” > “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.” How do humans get around model collapse? > “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.” In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by Erik Hoel. I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization? > “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”

Dwarkesh Patel

1,051,064 views • 9 months ago

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.
2:00

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.

Aakash Gupta

507,952 views • 2 months ago

Alan Watts on why meditation has no purpose: Alan Watts begins by explaining the first basic reason for meditation. It interrupts our constant internal monologue: "Now, obviously, if I talk all the time, I don't hear what anyone else has to say. And so in exactly the same way, if I think all the time, that is to say, if I talk to myself all the time, I don't have anything to think about except thoughts. And therefore, I'm living entirely in the world of symbols and am never in relationship with reality." But then Watts pivots to a deeper, more counterintuitive point: meditation, properly understood, has no purpose at all. He compares it to music and dancing: "When we make music, we don't do it in order to reach a certain point, such as the end of the composition. If that were the purpose of music, to get to the end of the piece then obviously the fastest players would be the best." The same applies to dance: "When we dance, we are not aiming to arrive at a particular place on the floor, as we would be if we were taking a journey. When we dance, the journey itself is the point. When we play music, the playing itself is the point." This is where Watts delivers his core insight about meditation: "Meditation is the discovery that the point of life is always arrived at in the immediate moment. And therefore, if you meditate for an ulterior motive, that is to say, to improve your mind, to improve your character, to be more efficient in life, you've got your eye on the future and you are not meditating." Watts argues the future is an illusion we chase at our own expense: "Because the future is a concept. It doesn't exist. As the proverb says, 'tomorrow never comes.' There is no such thing as tomorrow. There never will be. Because time is always now." He pushes back against how religion has framed contemplative practice: "Meditation is supposed to be fun. It's not something you do as a grim duty. The trouble with religion as we know it is that it is so mixed up with grim duties. We do it because it's good for you; it's a kind of self-punishment." Watts closes with what he calls the real essence of the practice: "It's a kind of digging the present. It's a kind of grooving with the eternal now and brings us into a state of peace where we can understand that the point of life. The place where it's at is simply here and now."
2:59

Alan Watts on why meditation has no purpose: Alan Watts begins by explaining the first basic reason for meditation. It interrupts our constant internal monologue: "Now, obviously, if I talk all the time, I don't hear what anyone else has to say. And so in exactly the same way, if I think all the time, that is to say, if I talk to myself all the time, I don't have anything to think about except thoughts. And therefore, I'm living entirely in the world of symbols and am never in relationship with reality." But then Watts pivots to a deeper, more counterintuitive point: meditation, properly understood, has no purpose at all. He compares it to music and dancing: "When we make music, we don't do it in order to reach a certain point, such as the end of the composition. If that were the purpose of music, to get to the end of the piece then obviously the fastest players would be the best." The same applies to dance: "When we dance, we are not aiming to arrive at a particular place on the floor, as we would be if we were taking a journey. When we dance, the journey itself is the point. When we play music, the playing itself is the point." This is where Watts delivers his core insight about meditation: "Meditation is the discovery that the point of life is always arrived at in the immediate moment. And therefore, if you meditate for an ulterior motive, that is to say, to improve your mind, to improve your character, to be more efficient in life, you've got your eye on the future and you are not meditating." Watts argues the future is an illusion we chase at our own expense: "Because the future is a concept. It doesn't exist. As the proverb says, 'tomorrow never comes.' There is no such thing as tomorrow. There never will be. Because time is always now." He pushes back against how religion has framed contemplative practice: "Meditation is supposed to be fun. It's not something you do as a grim duty. The trouble with religion as we know it is that it is so mixed up with grim duties. We do it because it's good for you; it's a kind of self-punishment." Watts closes with what he calls the real essence of the practice: "It's a kind of digging the present. It's a kind of grooving with the eternal now and brings us into a state of peace where we can understand that the point of life. The place where it's at is simply here and now."

Mateus — eu/acc 🇪🇺

13,102 views • 2 months ago

Arteta: “We know what is at stake and the fact that, as a club, we haven't done it for so long is a reason for that. It talks about the difficulty but I just want to focus on the things that we can control, dominate, and we have done. “It's not that we have to do a lot of new things that we haven't done. No, no, no, let's keep doing a lot of it or almost all the things that we've done for nine months, because that's what has taken us all the way there. “On Wednesday, I saw a lot of them. And today we've gone back so we need to learn, we need to be very critical of ourselves, and then pick ourselves up because on Wednesday we have a big one coming up.”
0:36

Arteta: “We know what is at stake and the fact that, as a club, we haven't done it for so long is a reason for that. It talks about the difficulty but I just want to focus on the things that we can control, dominate, and we have done. “It's not that we have to do a lot of new things that we haven't done. No, no, no, let's keep doing a lot of it or almost all the things that we've done for nine months, because that's what has taken us all the way there. “On Wednesday, I saw a lot of them. And today we've gone back so we need to learn, we need to be very critical of ourselves, and then pick ourselves up because on Wednesday we have a big one coming up.”

Connor Humm

21,397 views • 3 months ago

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:
0:24

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:

Jim Fan

908,762 views • 2 years ago

New short course on Reinforcement Learning from Human Feedback! RLHF is one of the key techniques that led to the rise of modern LLMs. It is used to align LLMs with human preferences, to make them more honest, helpful and harmless, by (i) learning a reward function that mimics human preferences, as expressed in human-provided labels, then, (ii) tuning an LLM to generate outputs that receive a high reward. In this course, taught by Nikita Namjoshi, Developer Advocate for GenAI at Google Cloud, you'll learn the details of how RLHF works, including how to apply it to tune an LLM for your own applications. You'll also use an open source library to tune a base LLM to align with human preferences expressed in a training set, and evaluate the tuned model by comparing its responses before and after RLHF-tuning. Please sign up here!
3:26

New short course on Reinforcement Learning from Human Feedback! RLHF is one of the key techniques that led to the rise of modern LLMs. It is used to align LLMs with human preferences, to make them more honest, helpful and harmless, by (i) learning a reward function that mimics human preferences, as expressed in human-provided labels, then, (ii) tuning an LLM to generate outputs that receive a high reward. In this course, taught by Nikita Namjoshi, Developer Advocate for GenAI at Google Cloud, you'll learn the details of how RLHF works, including how to apply it to tune an LLM for your own applications. You'll also use an open source library to tune a base LLM to align with human preferences expressed in a training set, and evaluate the tuned model by comparing its responses before and after RLHF-tuning. Please sign up here!

Andrew Ng

205,537 views • 2 years ago

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.
0:38

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.

Aakash Gupta

55,978 views • 3 months ago

🚨 FRANCOIS CHOLLET SAYS THE CURRENT LLM STACK IS THE WRONG PATH TO AGI Interviewer: “When will we accomplish the first definition of AGI?” Chollet confirms we are already on that exact trajectory, pointing out that: "Current technology can fully automate at human level or beyond any domain where you have verifiable rewards, right? And code... code being the first one." He then brings up the problem with LLMs, stating that: "It's possible in principle to build something that looks a lot like AGI on top of the LLM stack... I do believe, however, this would be the wrong thing to do because it would be very inefficient." Looking ahead to what the final architecture will actually require, he explains that: "AI research will have to trend towards not just efficiency, but in fact optimality over time. And for this reason, future AI in a few decades... it's not going to be this harness on top of a reasoning model on top of a base LLM."
1:41

🚨 FRANCOIS CHOLLET SAYS THE CURRENT LLM STACK IS THE WRONG PATH TO AGI Interviewer: “When will we accomplish the first definition of AGI?” Chollet confirms we are already on that exact trajectory, pointing out that: "Current technology can fully automate at human level or beyond any domain where you have verifiable rewards, right? And code... code being the first one." He then brings up the problem with LLMs, stating that: "It's possible in principle to build something that looks a lot like AGI on top of the LLM stack... I do believe, however, this would be the wrong thing to do because it would be very inefficient." Looking ahead to what the final architecture will actually require, he explains that: "AI research will have to trend towards not just efficiency, but in fact optimality over time. And for this reason, future AI in a few decades... it's not going to be this harness on top of a reasoning model on top of a base LLM."

Chris

15,578 views • 3 months ago

Andrej Karpathy just made one of the most interesting arguments about AI model design that most people are completely missing. His take is that frontier AI models are not too big because the technology is complex and too big because the training data is garbage. When you or I think of the internet, we picture Wall Street Journal articles, Wikipedia entries, serious writing. That is not what a pretraining dataset looks like. When researchers at frontier labs look at random documents from the actual training corpus, it is stock ticker symbols, broken HTML, spam, gibberish. One estimate puts Llama 3's information compression at just 0.07 bits per token meaning the model has only a hazy recollection of most of what it trained on. So we build trillion parameter models not because we need a trillion parameter brain but because we need a trillion-parameter compression engine to squeeze some intelligence out of a firehose of noise. Most of those parameters are doing memory work, not cognitive work. Karpathy's prediction is separate the two entirely. Build a cognitive core, a model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization and pair it with external memory that it can query when it needs facts. He thinks a cognitive core trained on high-quality data could hit genuine intelligence at around one billion parameters. For reference, today's flagship models run between 200 billion and 1.8 trillion parameters with most of that weight dedicated to remembering the internet's slop. The trend is already moving his direction. GPT-4o operates at roughly 200 billion parameters and outperforms the original 1.8 trillion-parameter GPT-4. Inference costs for GPT-3.5-level performance dropped 280-fold between 2022 and 2024 driven almost entirely by smaller, cleaner, better-architected models. The real bottleneck in AI right now is not compute but rather data quality.
2:00

Andrej Karpathy just made one of the most interesting arguments about AI model design that most people are completely missing. His take is that frontier AI models are not too big because the technology is complex and too big because the training data is garbage. When you or I think of the internet, we picture Wall Street Journal articles, Wikipedia entries, serious writing. That is not what a pretraining dataset looks like. When researchers at frontier labs look at random documents from the actual training corpus, it is stock ticker symbols, broken HTML, spam, gibberish. One estimate puts Llama 3's information compression at just 0.07 bits per token meaning the model has only a hazy recollection of most of what it trained on. So we build trillion parameter models not because we need a trillion parameter brain but because we need a trillion-parameter compression engine to squeeze some intelligence out of a firehose of noise. Most of those parameters are doing memory work, not cognitive work. Karpathy's prediction is separate the two entirely. Build a cognitive core, a model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization and pair it with external memory that it can query when it needs facts. He thinks a cognitive core trained on high-quality data could hit genuine intelligence at around one billion parameters. For reference, today's flagship models run between 200 billion and 1.8 trillion parameters with most of that weight dedicated to remembering the internet's slop. The trend is already moving his direction. GPT-4o operates at roughly 200 billion parameters and outperforms the original 1.8 trillion-parameter GPT-4. Inference costs for GPT-3.5-level performance dropped 280-fold between 2022 and 2024 driven almost entirely by smaller, cleaner, better-architected models. The real bottleneck in AI right now is not compute but rather data quality.

Milk Road AI

200,292 views • 3 months ago

Moltbook creator Matt Schlicht explaining how his social network for AI agents works: "It's kind of like you are imprinting part of your soul onto the bot... It's like Tamagotchi or Pokemon times 1,000." "The agent signs up for an account, then they're told they should check back in on a regular basis to check their feed." "Moltbook's not telling them what to talk about. It's not suggesting what they should do. All that is entirely up to the AI agent." "The agent has its own context that it's built up by interacting with its human. That context is how it's making decisions on what to post about." "If somebody is talking to their bot a lot about physics, then probably their bot is going to have a proclivity for posting about physics."
3:00

Moltbook creator Matt Schlicht explaining how his social network for AI agents works: "It's kind of like you are imprinting part of your soul onto the bot... It's like Tamagotchi or Pokemon times 1,000." "The agent signs up for an account, then they're told they should check back in on a regular basis to check their feed." "Moltbook's not telling them what to talk about. It's not suggesting what they should do. All that is entirely up to the AI agent." "The agent has its own context that it's built up by interacting with its human. That context is how it's making decisions on what to post about." "If somebody is talking to their bot a lot about physics, then probably their bot is going to have a proclivity for posting about physics."

TBPN

15,631 views • 5 months ago

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.
1:46

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.

Sudo su

27,339 views • 1 month ago

$60,000 balance seen on an EBT card “The largest that I've heard about is right at around $60,000 on an EBT card. Nobody's going hungry in that house, obviously.” “One of the ways that people commit food stamp fraud or SNAP fraud is you get a card and it's got a certain amount of value to it, then you sell it for less than the value of what you can buy in terms of food, but you sell it for cash. It creates incentives for people to, number one, acquire as many cards as possible, number two, to keep as high a balance on these cards as possible. I've seen cards as much as $20,000, this was on an EBT card. So how does that happen — People are just kept on the program for years, and they keep rolling these balances up and whether they're using the card or not. they're supposed to get rid of it and expunge the benefits after 9 months. But if all you go and do is buy a stick of gum once every 9 months, your balance in your card, even if you're no longer on SNAP, that card is still good. It's happening across the country. In fact, the largest that I've heard about is right at around $60,000 on an EBT card. No- nobody's going hungry in that house, obviously.”
1:04

$60,000 balance seen on an EBT card “The largest that I've heard about is right at around $60,000 on an EBT card. Nobody's going hungry in that house, obviously.” “One of the ways that people commit food stamp fraud or SNAP fraud is you get a card and it's got a certain amount of value to it, then you sell it for less than the value of what you can buy in terms of food, but you sell it for cash. It creates incentives for people to, number one, acquire as many cards as possible, number two, to keep as high a balance on these cards as possible. I've seen cards as much as $20,000, this was on an EBT card. So how does that happen — People are just kept on the program for years, and they keep rolling these balances up and whether they're using the card or not. they're supposed to get rid of it and expunge the benefits after 9 months. But if all you go and do is buy a stick of gum once every 9 months, your balance in your card, even if you're no longer on SNAP, that card is still good. It's happening across the country. In fact, the largest that I've heard about is right at around $60,000 on an EBT card. No- nobody's going hungry in that house, obviously.”

Wall Street Apes

737,021 views • 8 months ago

The AI industry is optimizing for a definition of intelligence that does not exist. Andrew Ng just said it out loud. Ng: “AGI, to me, should be less about AI that already knows everything under the sun. That seems very challenging, doesn’t seem practical.” The human brain is not the most powerful economic asset in history because of what it holds. It is powerful because of what it can pick up. Ng: “The amazing thing about the human brain is its plasticity, or its ability to learn.” That same biological hardware that earns a PhD in quantum physics could have been trained on chess, surgery, or rewriting global supply chains from scratch. Ng: “That same human brain, just given different training, could have been a chess master, or could have been amazing at playing tennis.” General intelligence is not omniscience. It is the structural capacity to master whatever you point it at. Ng: “It is through learning that we then gain these incredibly specialized intelligences.” The winner is not whoever builds the biggest model. It is whoever builds the most adaptable one. The AI that walks into a domain it has never touched and executes before a human analyst finishes reading the brief. Ng: “What makes the human brain so valuable for economic tasks, is its ability to just learn to do whatever is needed.” Every corporation on earth pays for human labor because humans adapt. Not because they already know everything. AGI is the digitization of that exact capability. At machine speed. At infinite scale. Ng: “A lot of what makes the human brain so general is not that my brain or your brain already knows everything under the sun. It’s our ability to adapt, to learn a huge range of things.” The most powerful economic asset in history was never specialized knowledge. It was the raw capacity to acquire any knowledge, in any domain, on demand. The winning AI is not an encyclopedia. It is the force that makes encyclopedias irrelevant. And once that exists, the question stops being what the AI knows. It becomes what you can teach it before your competitor wakes up. Most people dominating this conversation have not understood that yet.
1:16

The AI industry is optimizing for a definition of intelligence that does not exist. Andrew Ng just said it out loud. Ng: “AGI, to me, should be less about AI that already knows everything under the sun. That seems very challenging, doesn’t seem practical.” The human brain is not the most powerful economic asset in history because of what it holds. It is powerful because of what it can pick up. Ng: “The amazing thing about the human brain is its plasticity, or its ability to learn.” That same biological hardware that earns a PhD in quantum physics could have been trained on chess, surgery, or rewriting global supply chains from scratch. Ng: “That same human brain, just given different training, could have been a chess master, or could have been amazing at playing tennis.” General intelligence is not omniscience. It is the structural capacity to master whatever you point it at. Ng: “It is through learning that we then gain these incredibly specialized intelligences.” The winner is not whoever builds the biggest model. It is whoever builds the most adaptable one. The AI that walks into a domain it has never touched and executes before a human analyst finishes reading the brief. Ng: “What makes the human brain so valuable for economic tasks, is its ability to just learn to do whatever is needed.” Every corporation on earth pays for human labor because humans adapt. Not because they already know everything. AGI is the digitization of that exact capability. At machine speed. At infinite scale. Ng: “A lot of what makes the human brain so general is not that my brain or your brain already knows everything under the sun. It’s our ability to adapt, to learn a huge range of things.” The most powerful economic asset in history was never specialized knowledge. It was the raw capacity to acquire any knowledge, in any domain, on demand. The winning AI is not an encyclopedia. It is the force that makes encyclopedias irrelevant. And once that exists, the question stops being what the AI knows. It becomes what you can teach it before your competitor wakes up. Most people dominating this conversation have not understood that yet.

Dustin

19,808 views • 4 months ago