Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

A large portion of animal intelligence doesn't require any learning, claims Andrej Karpathy: it's baked into DNA. AI models, by contrast, start from random weights. They have to learn their intelligence, mostly by imitating the internet. This is so different that Andrej thinks it's a fundamentally different kind of... show more

Dwarkesh Patel

247,544 subscribers

82,164 views • 2 months ago •via X (Twitter)

Science & Technology Animals

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Andrej Karpathy beautifully explains the fundamental difference of learning between a human and an LLM. > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” many of the best minds in this field think LLMs are not really learning anything, and therefore are incapable of surpassing human-level intelligence.

Andrej Karpathy beautifully explains the fundamental difference of learning between a human and an LLM. > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” many of the best minds in this field think LLMs are not really learning anything, and therefore are incapable of surpassing human-level intelligence.

ℏεsam

895,224 views • 9 months ago

Andrej Karpathy in Stanford classroom said: "GPT is not a language model - It's a general-purpose computer" 90% of AI Twitter still thinks it's just autocomplete. It's not. >90% of the tools built on top of LLMs treat them like a smarter search engine. In this 40-minute lecture, Karpathy builds nanoGPT live 300 lines, reproduces GPT-2, explains every single decision. Worth more than any $500 deep learning bootcamp.

Andrej Karpathy in Stanford classroom said: "GPT is not a language model - It's a general-purpose computer" 90% of AI Twitter still thinks it's just autocomplete. It's not. >90% of the tools built on top of LLMs treat them like a smarter search engine. In this 40-minute lecture, Karpathy builds nanoGPT live 300 lines, reproduces GPT-2, explains every single decision. Worth more than any $500 deep learning bootcamp.

Ricker

117,471 views • 20 days ago

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Dwarkesh Patel

55,137 views • 1 month ago

this was the most important bit of the ilya interview imo. humans are not AGI, and LLMs won’t become like humans with more scaling. humans lack a huge amount of knowledge and have insanely good “continual learning” powered by extremely good reward functions (except for addiction) that guide them. and so it’s more about the learning rather than knowledge. the term AGI was coined as a response to the narrow AI which was only capable of intelligence in a specific domain (chess, game, code, etc) but General intelligence is able to learn different topics because that’s what “learning” is about. pre-training was thought to be the solution to reaching AGI because you train AI on so many different topics. but they have “overshot” that target. amazing perspective.

this was the most important bit of the ilya interview imo. humans are not AGI, and LLMs won’t become like humans with more scaling. humans lack a huge amount of knowledge and have insanely good “continual learning” powered by extremely good reward functions (except for addiction) that guide them. and so it’s more about the learning rather than knowledge. the term AGI was coined as a response to the narrow AI which was only capable of intelligence in a specific domain (chess, game, code, etc) but General intelligence is able to learn different topics because that’s what “learning” is about. pre-training was thought to be the solution to reaching AGI because you train AI on so many different topics. but they have “overshot” that target. amazing perspective.

ℏεsam

53,444 views • 8 months ago

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)

Rohan Paul

440,435 views • 15 days ago

TESLA ISN'T JUST AHEAD IN AI, IT'S AN ORDER OF MAGNITUDE MORE EFFICIENT THAN THE REST OF THE WORLD "I actually think Tesla is ahead of the rest of the world in intelligence density of AI by an order of magnitude or more This is going to sound like a pretty bold statement, but I know what the intelligence efficiency of the big models are like, Grok and a bunch of the other models Tesla’s AI is, in terms of its memory efficiency, I think more than an order of magnitude better. The Tesla AI is very compute-efficient and very memory-efficient One of the metrics one should consider for any given AI model is the intelligence per gigabyte. Especially when you’re constrained on RAM, having an AI that has very high intelligence density per gigabyte for a given number of gigabytes, how much functionality can you get out of it" - Elon Musk

TESLA ISN'T JUST AHEAD IN AI, IT'S AN ORDER OF MAGNITUDE MORE EFFICIENT THAN THE REST OF THE WORLD "I actually think Tesla is ahead of the rest of the world in intelligence density of AI by an order of magnitude or more This is going to sound like a pretty bold statement, but I know what the intelligence efficiency of the big models are like, Grok and a bunch of the other models Tesla’s AI is, in terms of its memory efficiency, I think more than an order of magnitude better. The Tesla AI is very compute-efficient and very memory-efficient One of the metrics one should consider for any given AI model is the intelligence per gigabyte. Especially when you’re constrained on RAM, having an AI that has very high intelligence density per gigabyte for a given number of gigabytes, how much functionality can you get out of it" - Elon Musk

X Freeze

62,736 views • 6 months ago

Okay, this is actually insane Recent UnitedHealthcare lawsuit reveals that the company KNOWINGLY has been using faulty artificial intelligence to deny claims to increase profitability. The AI they’ve been using HAS A 90% INACCURACY RATE “The company knowingly used artificial intelligence. That that was faulty. That's where this comes out of. It comes out of litigation” “Separate litigation that they knowingly used faulty artificial intelligence to deny claims that were legitimate to increase profitability. That is a big deal. — This is very, very overt artificial intelligence tool with 90% inaccuracy” “It's a sinister thing.”

Okay, this is actually insane Recent UnitedHealthcare lawsuit reveals that the company KNOWINGLY has been using faulty artificial intelligence to deny claims to increase profitability. The AI they’ve been using HAS A 90% INACCURACY RATE “The company knowingly used artificial intelligence. That that was faulty. That's where this comes out of. It comes out of litigation” “Separate litigation that they knowingly used faulty artificial intelligence to deny claims that were legitimate to increase profitability. That is a big deal. — This is very, very overt artificial intelligence tool with 90% inaccuracy” “It's a sinister thing.”

Wall Street Apes

461,093 views • 1 year ago

ANDREJ KARPATHY: "the internet is terrible. A huge amount of slop and garbage from all the corners of the internet." Type 1: you ask generic Claude generic questions, you get back generic internet-trained answers. Type 2: you brief Claude with your business, your voice, your audience. Same model. Different fluency. The gap between the two types isn't intelligence. It's setup. Bookmark this for the next time someone says Claude is generic and can't replace a real specialist. The article below has the 7 specific setups that turn the internet-slop model into a specialist for your work.

ANDREJ KARPATHY: "the internet is terrible. A huge amount of slop and garbage from all the corners of the internet." Type 1: you ask generic Claude generic questions, you get back generic internet-trained answers. Type 2: you brief Claude with your business, your voice, your audience. Same model. Different fluency. The gap between the two types isn't intelligence. It's setup. Bookmark this for the next time someone says Claude is generic and can't replace a real specialist. The article below has the 7 specific setups that turn the internet-slop model into a specialist for your work.

Zephyr

147,644 views • 2 months ago

.𝔊𝔴𝔢𝔯𝔫 almost thinks there's no such thing as general intelligence. Humans and AIs just learn a large number of individual specialized tricks. In any given situation we're doing search over special cases, nothing more. What matters is just the number of individual tricks that we can search over - which is mostly determined by compute.

.𝔊𝔴𝔢𝔯𝔫 almost thinks there's no such thing as general intelligence. Humans and AIs just learn a large number of individual specialized tricks. In any given situation we're doing search over special cases, nothing more. What matters is just the number of individual tricks that we can search over - which is mostly determined by compute.

Dwarkesh Patel

54,130 views • 2 months ago

Yann LeCun says you cannot build a reliable agentic system without a world model LLMs don't have world models. They can't predict the consequences of their actions before taking them "they just act, and whatever happens next is someone else's problem" Without that, it's not intelligence

Yann LeCun says you cannot build a reliable agentic system without a world model LLMs don't have world models. They can't predict the consequences of their actions before taking them "they just act, and whatever happens next is someone else's problem" Without that, it's not intelligence

Haider.

332,446 views • 2 months ago

Barack Obama made using propaganda against Americans legal in the United States. Restore The Smith-Mundt Act “So a hundred different journalists are being paid directly by the CIA. They had CIA intelligence officers, assets, CIA agents just working hand in glove inside of newsrooms, writing articles, producing the news. Operation Mockingbird. Nobody's ever asked CIA about this in modern times” “Tulsi Gabbard confirming that Operation Mockingbird is still ongoing, which what she told me was that, yes, this is still happening and that we are fighting to this day. Intel operatives that are running disinformation campaigns through the corporate press.” “Within this administration, where there are people within the intelligence community who believe that their will is more important than the will of the American people and will weaponize intelligence by leaking it to their friends within the mainstream media with the intent of undermining President Trump's agenda” “The intelligence world was allying itself with journalists. Yeah, this is still ongoing. It's a massive, massive problem and it's something that the administration is currently dealing with”

Barack Obama made using propaganda against Americans legal in the United States. Restore The Smith-Mundt Act “So a hundred different journalists are being paid directly by the CIA. They had CIA intelligence officers, assets, CIA agents just working hand in glove inside of newsrooms, writing articles, producing the news. Operation Mockingbird. Nobody's ever asked CIA about this in modern times” “Tulsi Gabbard confirming that Operation Mockingbird is still ongoing, which what she told me was that, yes, this is still happening and that we are fighting to this day. Intel operatives that are running disinformation campaigns through the corporate press.” “Within this administration, where there are people within the intelligence community who believe that their will is more important than the will of the American people and will weaponize intelligence by leaking it to their friends within the mainstream media with the intent of undermining President Trump's agenda” “The intelligence world was allying itself with journalists. Yeah, this is still ongoing. It's a massive, massive problem and it's something that the administration is currently dealing with”

Wall Street Apes

159,311 views • 1 year ago

BREAKING: Andrej Karpathy calls out Sam Altman Altman: >"We are now confident we know how to build AGI" >"2025 is the year of AI agents" Karpathy: >"I was triggered by that over-prediction" >"More accurately, it's the decade of agents" >"There's SO much work to be done" Hmm looks like Sam Altman has been lying to investors all year to raise trillions… The bubble is popping.

BREAKING: Andrej Karpathy calls out Sam Altman Altman: >"We are now confident we know how to build AGI" >"2025 is the year of AI agents" Karpathy: >"I was triggered by that over-prediction" >"More accurately, it's the decade of agents" >"There's SO much work to be done" Hmm looks like Sam Altman has been lying to investors all year to raise trillions… The bubble is popping.

NIK

579,241 views • 9 months ago

Here’s How 1 McDonald’s Hamburger Contains 1300 Strands Of Different DNA. This Is The Result Of Going From Small Farms To Mega Corporations “We're pooling the bacteria from a thousand different animals” “Thousands of animals” contribute “to a single hamburger.” “This is a combo bin of meat. The meat in these boxes comes from many different animals and many different places. Some is even from overseas. All of it is ground together to make the juicy hamburger Americans love. Hamburger used to be the scraps left over from butchering just one cow.” But a significant change has occurred. Now parts of many cows are blended together by the ton. ‌ If we take the meat from one animal, grind it up and make ground beef from just one animal, we're including only the bacteria from one animal. But if we take the meat from a thousand different animals and grind that together, we're pooling the bacteria from a thousand different animals as well. So explain this to me, how many different cows would be in one burger? Well, I suspect that there are hundreds or even thousands of animals that have contributed to a single hamburger.”

Here’s How 1 McDonald’s Hamburger Contains 1300 Strands Of Different DNA. This Is The Result Of Going From Small Farms To Mega Corporations “We're pooling the bacteria from a thousand different animals” “Thousands of animals” contribute “to a single hamburger.” “This is a combo bin of meat. The meat in these boxes comes from many different animals and many different places. Some is even from overseas. All of it is ground together to make the juicy hamburger Americans love. Hamburger used to be the scraps left over from butchering just one cow.” But a significant change has occurred. Now parts of many cows are blended together by the ton. ‌ If we take the meat from one animal, grind it up and make ground beef from just one animal, we're including only the bacteria from one animal. But if we take the meat from a thousand different animals and grind that together, we're pooling the bacteria from a thousand different animals as well. So explain this to me, how many different cows would be in one burger? Well, I suspect that there are hundreds or even thousands of animals that have contributed to a single hamburger.”

Wall Street Apes

3,114,142 views • 2 years ago

Roger Penrose, Nobel Prize-winning physicist and mathematician, explains why we should stop calling it AI and start calling it "artificial cleverness": He believes the entire field is mislabelled, and the label itself is doing damage. His objection is simple but cuts deep: "The name is wrong. It's not artificial intelligence. It's not intelligence. Intelligence would involve consciousness. Well, if it's a machine, it's not conscious." For Penrose, people have confused raw computing power with genuine understanding. "People have lost the plot. They've lost it in the power of computing. The thing is that computers have got so powerful that they've lost the thread of what they're doing. But I think consciousness is something different. It's not computational." He believes the term itself has hypnotized people into a category error: "People are so hypnotized. The trouble is that AI is a bad term. It means artificial intelligence. Now intelligence in my view is conscious. That's what intelligence is about." So he proposes a rename. Artificial Cleverness. AC instead of AI. To illustrate the distinction, Penrose draws on his experience teaching mathematics: "You have mathematics students. Some of them understand what they're doing. Some are just clever. They can repeat what they've learned. They know how to do it very cleverly. They can calculate very well, but they don't necessarily understand what they're doing." That gap, between calculating well and actually understanding, is the gap Penrose sees between today's machines and genuine intelligence. Cleverness can be manufactured. Consciousness, in his view, cannot. So the question worth sitting with: when we call a system "intelligent," are we describing what it does, or quietly assuming something about what it is?

Roger Penrose, Nobel Prize-winning physicist and mathematician, explains why we should stop calling it AI and start calling it "artificial cleverness": He believes the entire field is mislabelled, and the label itself is doing damage. His objection is simple but cuts deep: "The name is wrong. It's not artificial intelligence. It's not intelligence. Intelligence would involve consciousness. Well, if it's a machine, it's not conscious." For Penrose, people have confused raw computing power with genuine understanding. "People have lost the plot. They've lost it in the power of computing. The thing is that computers have got so powerful that they've lost the thread of what they're doing. But I think consciousness is something different. It's not computational." He believes the term itself has hypnotized people into a category error: "People are so hypnotized. The trouble is that AI is a bad term. It means artificial intelligence. Now intelligence in my view is conscious. That's what intelligence is about." So he proposes a rename. Artificial Cleverness. AC instead of AI. To illustrate the distinction, Penrose draws on his experience teaching mathematics: "You have mathematics students. Some of them understand what they're doing. Some are just clever. They can repeat what they've learned. They know how to do it very cleverly. They can calculate very well, but they don't necessarily understand what they're doing." That gap, between calculating well and actually understanding, is the gap Penrose sees between today's machines and genuine intelligence. Cleverness can be manufactured. Consciousness, in his view, cannot. So the question worth sitting with: when we call a system "intelligent," are we describing what it does, or quietly assuming something about what it is?

Big Brain AI

117,778 views • 2 months ago

Stanley Druckenmiller: My advantage is not IQ. It's trigger pulling. I admit it's some kind of intelligence. I wasn't in the top 10% of my class. A lot people think i'm smarter than I am because i'm good at our business. But I have a very narrow form of intelligence that allows me to love and play this game

Stanley Druckenmiller: My advantage is not IQ. It's trigger pulling. I admit it's some kind of intelligence. I wasn't in the top 10% of my class. A lot people think i'm smarter than I am because i'm good at our business. But I have a very narrow form of intelligence that allows me to love and play this game

Bourbon Capital

219,675 views • 5 months ago

Continual learning sometimes gets discussed as if the goal is to dissolve the context/weights distinction. Let the model just keep accumulating, fine-tuning itself on the fly. Andrej Karpathy points out, though, that this isn't how humans do it. Our working memory gets wiped regularly. What we actually have is a consolidation process (sleep) that distills stuff into the brain, in a weird and lossy way. This is very different from how people sometimes talk about continual learning. It's not obvious it's something you can get for free from doing long enough RL loops.

Continual learning sometimes gets discussed as if the goal is to dissolve the context/weights distinction. Let the model just keep accumulating, fine-tuning itself on the fly. Andrej Karpathy points out, though, that this isn't how humans do it. Our working memory gets wiped regularly. What we actually have is a consolidation process (sleep) that distills stuff into the brain, in a weird and lossy way. This is very different from how people sometimes talk about continual learning. It's not obvious it's something you can get for free from doing long enough RL loops.

Dwarkesh Patel

59,993 views • 2 months ago

🚨The United States is Led by a GANGSTER GROUP Jeffrey Sachs: The Washington Post, which is the mouthpiece of the intelligence agencies, twists it so that it's Putin that's the villain because he doesn't respond to Trump's thuggery. So Putin's a liar because he doesn't stand up to Trump's thuggery, rather than an article that Trump is a thug and maybe it's a, it's a dangerous thing for the world that United States is led by a gangster, group.

🚨The United States is Led by a GANGSTER GROUP Jeffrey Sachs: The Washington Post, which is the mouthpiece of the intelligence agencies, twists it so that it's Putin that's the villain because he doesn't respond to Trump's thuggery. So Putin's a liar because he doesn't stand up to Trump's thuggery, rather than an article that Trump is a thug and maybe it's a, it's a dangerous thing for the world that United States is led by a gangster, group.

Ignorance, the root and stem of all evil

46,230 views • 5 months ago

Andrej Karpathy called the current AI models as "slop" "I do think that overall the models are not there yet. I feel like the industry is making too big of a jump and trying to pretend that this is amazing, but it's not. It's slop, and I think they are not coming to terms with it. Maybe they are trying to fundraise or something like that, I'm not sure what's going on. We are at this intermediate stage. The models are amazing, but they still need a lot of work for now. Autocomplete is my sweet spot." --- On the latest Dwarkesh Patel podcast.

Andrej Karpathy called the current AI models as "slop" "I do think that overall the models are not there yet. I feel like the industry is making too big of a jump and trying to pretend that this is amazing, but it's not. It's slop, and I think they are not coming to terms with it. Maybe they are trying to fundraise or something like that, I'm not sure what's going on. We are at this intermediate stage. The models are amazing, but they still need a lot of work for now. Autocomplete is my sweet spot." --- On the latest Dwarkesh Patel podcast.

Rohan Paul

33,331 views • 9 months ago

Jeremy Corbell says he’s seen the ‘disc UFO in the clouds’ video taken from space by satellite that’s described by Matthew Brown. “That is a piece of footage that within the Intelligence Community, if they have access to the servers, is kind of famous at this point.”

Jeremy Corbell says he’s seen the ‘disc UFO in the clouds’ video taken from space by satellite that’s described by Matthew Brown. “That is a piece of footage that within the Intelligence Community, if they have access to the servers, is kind of famous at this point.”

UAP James

209,327 views • 1 year ago