Video wird geladen...
Video konnte nicht geladen werden
A key argument for short timelines is that LLMs are basically baby AGIs who only need some small unhobblings in order to match humans. But why is this argument any different than saying AlphaZero is baby AGI? After all, it was superhuman in some domains, genuinely creative (remember move... show more
46,667 Aufrufe • vor 1 Jahr •via X (Twitter)
11 Kommentare
LLMs per se are very poor at certain games (poker, chess, I've heard even Wordle). But the same games are ~solved if you train a model on them. Probably better to assert AGI can be achieved through machine learning broadly rather than thru one-shotting.
Logic dictates that something—or someone—always had to exist. Assume it was a “someone,” not a “something.” Why would such a being create a world like ours, one filled with pain? The Advent of Time provides a definitive answer.
The unhobblings aren't small [ v ] Understanding vision, speech and language [ v ] Fuzzy reasoning [ ] Exact reasoning [ ] Learning from experience My timelines are short, but those are major shortcomings.
Yeah, I made a similar point to an OpenAI employee. Games are helpful for understanding information symmetry and actions under particular conditions (like with game theory), but if AGI is intended to exceed human cognition, learn and self-improve autonomously, or plan and execute goals, then those are different scenarios than games with finite moves or deterministic outcomes.
Language encodes enough information about the world, and how to reason about that information, that it would be very strange to me if there is some other capability that is needed to generalize to intelligence
calling LLMs baby AGIs is like saying toddlers can run marathons with a little encouragement. alpha zero is impressive, but it's still a game piece, not a player. let’s keep our eyes peeled for the real game changers lurking in the shadows.
AGI is a fluid concept that varies by individual. More useful benchmarks are beating humans at specific tasks such as playing go or driving cars. Machines have been doing this since dawn of humanity and it is nothing new.
The big labs tried to build dedicated video game playing AIs for Dota and StarCraft. And they made very strong models, but they never achieved the kind of dominance we saw with AlphaGo (that decisively beat the best humans in the world) and then AlphaZero leapfrogged it and basically "solved" go, at least compared to human abilities. The best strategy game AIs had some kinds of limitations on human okay, or arguably used inhumanly fast button pressing to win in key areas, or just never decisively beat all the best human players. AlphaStar played online in 2019, years before chatGPT and years before the explosion in compute power we have available now. Why haven't we seen anyone design and train an AI that can do for StarCraft what AlphaZero did for Go?
AGI timelines mirror liquidity cycles?
LLMs are getting better simultaneously at a very broad range of tasks with increased size. AlphaZero type systems never had that broad range.
The evolution of LLMs is actually very weird. Nothing like biology.
![The most interesting part for me is where Andrej Karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.” A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer. > “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.” So what do humans do instead? > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” > “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.” Why can’t we just add this training to LLMs today? > “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.” > “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.” > “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.” How do humans get around model collapse? > “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.” In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by Erik Hoel. I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization? > “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”](https://image.24vids.com/tw-1979259041013731752/amplify_video_thumb/1979257571786543104/img/OqUh56yit9gUQWnb.jpg)
