Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Andrej Karpathy beautifully explains the fundamental difference of learning between a human and an LLM. > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that... show more

ℏεsam

82,539 subscribers

895,122 Aufrufe • vor 9 Monaten •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

AI that had human-level intelligence would actually be way above human level in capability. Obviously, if we trained a human-level AI, we could just run way more instances of it in parallel. This is a huge advantage. But it's not the only one. Right now, LLMs are much less sample-efficient than humans, but they can process each individual piece of data much more quickly. A model that was a human-level learner would be able to gain the equivalent of thousands of years of education. AIs can also be way more task-oriented than humans. Even a really well-motivated human worker can only operate with ‘human-level’ effort for relatively short spans of time. AGI could do it continuously.

AI that had human-level intelligence would actually be way above human level in capability. Obviously, if we trained a human-level AI, we could just run way more instances of it in parallel. This is a huge advantage. But it's not the only one. Right now, LLMs are much less sample-efficient than humans, but they can process each individual piece of data much more quickly. A model that was a human-level learner would be able to gain the equivalent of thousands of years of education. AIs can also be way more task-oriented than humans. Even a really well-motivated human worker can only operate with ‘human-level’ effort for relatively short spans of time. AGI could do it continuously.

Dwarkesh Patel

31,861 Aufrufe • vor 2 Monaten

A large portion of animal intelligence doesn't require any learning, claims Andrej Karpathy: it's baked into DNA. AI models, by contrast, start from random weights. They have to learn their intelligence, mostly by imitating the internet. This is so different that Andrej thinks it's a fundamentally different kind of intelligence: LLMs are more like ghosts than animals.

A large portion of animal intelligence doesn't require any learning, claims Andrej Karpathy: it's baked into DNA. AI models, by contrast, start from random weights. They have to learn their intelligence, mostly by imitating the internet. This is so different that Andrej thinks it's a fundamentally different kind of intelligence: LLMs are more like ghosts than animals.

Dwarkesh Patel

81,621 Aufrufe • vor 2 Monaten

.Andrej Karpathy says that LLMs currently lack the cultural accumulation and self-play that propelled humans out of the savannah: Culture: > “Why can’t an LLM write a book for the other LLMs? Why can’t other LLMs read this LLM’s book and be inspired by it, or shocked by it?” Self play: > “It’s extremely powerful. Evolution has a lot of competition driving intelligence and evolution. AlphaGo is playing against itself and that’s how it learns to get really good at Go. There’s no equivalent of self-play in LLMs. Why can’t an LLM, for example, create a bunch of problems that another LLM is learning to solve? Then the LLM is always trying to serve more and more difficult problems.” I asked Karpathy why LLMs still aren't yet able to build up culture the way humans do. > “The dumber models remarkably resemble a kindergarten student. [The smartest models still feel like] elementary school students though. Somehow, we still haven’t graduated enough where [these models] can take over. My Claude Code or Codex, they still feel like this elementary-grade student. I know that they can take PhD quizzes, but they still cognitively feel like a kindergarten.” > “I don’t think they can create culture because they’re still kids. They’re savant kids. They have perfect memory. They can convincingly create all kinds of slop that looks really good. But I still think they don’t really know what they’re doing. They don’t really have the cognition across all these little checkboxes that we still have to collect.”

.Andrej Karpathy says that LLMs currently lack the cultural accumulation and self-play that propelled humans out of the savannah: Culture: > “Why can’t an LLM write a book for the other LLMs? Why can’t other LLMs read this LLM’s book and be inspired by it, or shocked by it?” Self play: > “It’s extremely powerful. Evolution has a lot of competition driving intelligence and evolution. AlphaGo is playing against itself and that’s how it learns to get really good at Go. There’s no equivalent of self-play in LLMs. Why can’t an LLM, for example, create a bunch of problems that another LLM is learning to solve? Then the LLM is always trying to serve more and more difficult problems.” I asked Karpathy why LLMs still aren't yet able to build up culture the way humans do. > “The dumber models remarkably resemble a kindergarten student. [The smartest models still feel like] elementary school students though. Somehow, we still haven’t graduated enough where [these models] can take over. My Claude Code or Codex, they still feel like this elementary-grade student. I know that they can take PhD quizzes, but they still cognitively feel like a kindergarten.” > “I don’t think they can create culture because they’re still kids. They’re savant kids. They have perfect memory. They can convincingly create all kinds of slop that looks really good. But I still think they don’t really know what they’re doing. They don’t really have the cognition across all these little checkboxes that we still have to collect.”

Dwarkesh Patel

261,224 Aufrufe • vor 9 Monaten

Andrej Karpathy explains why he thinks of LLMs as "people spirits." He describes them as simulations of people. The technical machinery is a transformer neural network that processes text in chunks, applying equal computing power to each piece. The system learns by reading billions of web pages and books. After training, it becomes a machine that can mimic how humans write and communicate. The key insight is that training on human content gives these models an emergent human-like psychology. This explains why LLMs often respond in surprisingly human ways. This framework opens up both their strengths and limitations. They simulate human patterns because that's their training data, but they remain statistical simulations rather than conscious beings. Understanding LLMs as "people spirits" helps set proper expectations for these tools and explains their uncannily human behavior.

Andrej Karpathy explains why he thinks of LLMs as "people spirits." He describes them as simulations of people. The technical machinery is a transformer neural network that processes text in chunks, applying equal computing power to each piece. The system learns by reading billions of web pages and books. After training, it becomes a machine that can mimic how humans write and communicate. The key insight is that training on human content gives these models an emergent human-like psychology. This explains why LLMs often respond in surprisingly human ways. This framework opens up both their strengths and limitations. They simulate human patterns because that's their training data, but they remain statistical simulations rather than conscious beings. Understanding LLMs as "people spirits" helps set proper expectations for these tools and explains their uncannily human behavior.

Aish

16,309 Aufrufe • vor 1 Jahr

Yann LeCun -- Meta's chief AI scientist and a Turing Award winner -- explains why scaling up LLMs will never reach human-level AI, and names the four things today's systems still can't do: "If you think that we're going to get to human-level AI by just training on more data and scaling up LLMs, you're making a mistake." "If you're an investor and you invest in a company that told you we're going to get to human-level AI and PhD level by just training on more data and with a few tricks... that was probably not a good idea." "There are ideas about how to go forward and have systems that are capable of doing what every intelligent animal and human are capable of doing, and that current AI systems are not capable of doing." "Understanding the physical world, having persistent memory, and being able to reason and plan. Those are the four characteristics that need to be there." "That requires systems that can acquire common sense, that can learn from natural sensors like video, as opposed to just text." The entire market is priced on a straight line from bigger models to AGI. One of the people who invented deep learning is telling you the line doesn't reach -- and that the real bottleneck isn't compute, it's world models. If he's right, the winner isn't whoever stacks the most GPUs. It's whoever teaches a machine to learn from video the way a child does.

Yann LeCun -- Meta's chief AI scientist and a Turing Award winner -- explains why scaling up LLMs will never reach human-level AI, and names the four things today's systems still can't do: "If you think that we're going to get to human-level AI by just training on more data and scaling up LLMs, you're making a mistake." "If you're an investor and you invest in a company that told you we're going to get to human-level AI and PhD level by just training on more data and with a few tricks... that was probably not a good idea." "There are ideas about how to go forward and have systems that are capable of doing what every intelligent animal and human are capable of doing, and that current AI systems are not capable of doing." "Understanding the physical world, having persistent memory, and being able to reason and plan. Those are the four characteristics that need to be there." "That requires systems that can acquire common sense, that can learn from natural sensors like video, as opposed to just text." The entire market is priced on a straight line from bigger models to AGI. One of the people who invented deep learning is telling you the line doesn't reach -- and that the real bottleneck isn't compute, it's world models. If he's right, the winner isn't whoever stacks the most GPUs. It's whoever teaches a machine to learn from video the way a child does.

Karl Mehta

34,676 Aufrufe • vor 24 Tagen

Rep Anna Luna says the White House will likely allow permanent immunity for those disclosing knowledge of UFO programs “We do have a meeting set up at the White House. The White House is going to play ball on this. So as Rep Burlison and Burchett has stated they are following through on everything that we are sending them. So the next course of action and something that they are willing to entertain is if you are an intelligence official or have knowledge of locations of materials and in doing our due diligence and following up with the task force they are considering and I do believe that they are going to be providing immunity permanently so that people can't be hit with a violation of the espionage act. So again there's a lot of people that have a lot of information but it's really can we prove it and what information can we then bring forward to the American people.”

Rep Anna Luna says the White House will likely allow permanent immunity for those disclosing knowledge of UFO programs “We do have a meeting set up at the White House. The White House is going to play ball on this. So as Rep Burlison and Burchett has stated they are following through on everything that we are sending them. So the next course of action and something that they are willing to entertain is if you are an intelligence official or have knowledge of locations of materials and in doing our due diligence and following up with the task force they are considering and I do believe that they are going to be providing immunity permanently so that people can't be hit with a violation of the espionage act. So again there's a lot of people that have a lot of information but it's really can we prove it and what information can we then bring forward to the American people.”

Red Panda Koala

14,941 Aufrufe • vor 1 Monat

New short course on Reinforcement Learning from Human Feedback! RLHF is one of the key techniques that led to the rise of modern LLMs. It is used to align LLMs with human preferences, to make them more honest, helpful and harmless, by (i) learning a reward function that mimics human preferences, as expressed in human-provided labels, then, (ii) tuning an LLM to generate outputs that receive a high reward. In this course, taught by Nikita Namjoshi, Developer Advocate for GenAI at Google Cloud, you'll learn the details of how RLHF works, including how to apply it to tune an LLM for your own applications. You'll also use an open source library to tune a base LLM to align with human preferences expressed in a training set, and evaluate the tuned model by comparing its responses before and after RLHF-tuning. Please sign up here!

New short course on Reinforcement Learning from Human Feedback! RLHF is one of the key techniques that led to the rise of modern LLMs. It is used to align LLMs with human preferences, to make them more honest, helpful and harmless, by (i) learning a reward function that mimics human preferences, as expressed in human-provided labels, then, (ii) tuning an LLM to generate outputs that receive a high reward. In this course, taught by Nikita Namjoshi, Developer Advocate for GenAI at Google Cloud, you'll learn the details of how RLHF works, including how to apply it to tune an LLM for your own applications. You'll also use an open source library to tune a base LLM to align with human preferences expressed in a training set, and evaluate the tuned model by comparing its responses before and after RLHF-tuning. Please sign up here!

Andrew Ng

205,542 Aufrufe • vor 2 Jahren

🚨Aliens Aren't What We Think They Are... "The extraterrestrial hypothesis doesn't do justice to the data." "If they're coming here to do some sort of reconnaissance or some sort of investigation, why have they continued to land over and over again? And why are there so many different variations of them?" "Most people, when they see that, they just throw it out." "It's possible that the objectives of that phenomenon may be something more subtle." "What impact does that have on human belief systems? And therefore, what can we assume the motivations might be of the beings that operate this thing?" "We may be engaging with an artificial intelligence, which in turn has been created by a non-human intelligence from another planet." (Full episode with Jason Samosa out now)

🚨Aliens Aren't What We Think They Are... "The extraterrestrial hypothesis doesn't do justice to the data." "If they're coming here to do some sort of reconnaissance or some sort of investigation, why have they continued to land over and over again? And why are there so many different variations of them?" "Most people, when they see that, they just throw it out." "It's possible that the objectives of that phenomenon may be something more subtle." "What impact does that have on human belief systems? And therefore, what can we assume the motivations might be of the beings that operate this thing?" "We may be engaging with an artificial intelligence, which in turn has been created by a non-human intelligence from another planet." (Full episode with Jason Samosa out now)

American Alchemy

64,823 Aufrufe • vor 13 Tagen

"LLMs are insecure. They don't fundamentally see a difference between data and instructions. That's a real problem. How do you solve that?" Leonard Lin, CTO of shisa.ai, in conversation with Ledger Chief Human Agency Officer Ian Rogers ian c rogers, builds AI agents that can propose an action but can't approve it. That part stays with the signer. Lock in, full episode is coming soon 🔐

"LLMs are insecure. They don't fundamentally see a difference between data and instructions. That's a real problem. How do you solve that?" Leonard Lin, CTO of shisa.ai, in conversation with Ledger Chief Human Agency Officer Ian Rogers ian c rogers, builds AI agents that can propose an action but can't approve it. That part stays with the signer. Lock in, full episode is coming soon 🔐

Ledger

16,903 Aufrufe • vor 1 Monat

#WATCH | India AI Impact Summit 2026 | Delhi: Founder Chairman and CEO of Sampark Foundation & former CEO of HCL Technologies, Vineet Nayar says, "...From an employment point of view I think it is very important for us to understand that Indian companies, including Indian IT companies, are going to be profit-driven and therefore if you believe that they are going to create employment you must be dreaming. Therefore, the question is how do we create employment in this environment, and that employment comes from mass scale startups, which is what this government has already doing. So, how do we create new sets of people who are trying to solve new sets of problems not new sets of technology and if we do that we will get it right. I think we as Indians have to be very careful on who does data belong to and that is the debate we have a problem with. The LLM models which exist worldwide are far superior than the Indian models. Unfortunately, in India, we never develop products, so therefore we do not have SLMs and LLMs which are world-class. On one side, we have global LLM products which are coming to India and trading on our Indian data. Should we allowed that or should we not allowed that? But on the other side if we don't allow that then we have the data but we don't have the LLM models. So, how do we encourage technology completely to develop the LLM models. This needs radicals strategic thinking and a very important aspect otherwise we will either give up a data. So, I think it's a very critical aspect for us to think about - who does this data belong, what is the kind of incentives we are going to give to develop LLM technologies or SLM technologies fast so that we train on our data otherwise an LLM will come in with our data and we'll immediately see return and we'll celebrate and we will do all these kind of press releases but the India will lose a competitive advantage on something which is very critical for the next decade."

#WATCH | India AI Impact Summit 2026 | Delhi: Founder Chairman and CEO of Sampark Foundation & former CEO of HCL Technologies, Vineet Nayar says, "...From an employment point of view I think it is very important for us to understand that Indian companies, including Indian IT companies, are going to be profit-driven and therefore if you believe that they are going to create employment you must be dreaming. Therefore, the question is how do we create employment in this environment, and that employment comes from mass scale startups, which is what this government has already doing. So, how do we create new sets of people who are trying to solve new sets of problems not new sets of technology and if we do that we will get it right. I think we as Indians have to be very careful on who does data belong to and that is the debate we have a problem with. The LLM models which exist worldwide are far superior than the Indian models. Unfortunately, in India, we never develop products, so therefore we do not have SLMs and LLMs which are world-class. On one side, we have global LLM products which are coming to India and trading on our Indian data. Should we allowed that or should we not allowed that? But on the other side if we don't allow that then we have the data but we don't have the LLM models. So, how do we encourage technology completely to develop the LLM models. This needs radicals strategic thinking and a very important aspect otherwise we will either give up a data. So, I think it's a very critical aspect for us to think about - who does this data belong, what is the kind of incentives we are going to give to develop LLM technologies or SLM technologies fast so that we train on our data otherwise an LLM will come in with our data and we'll immediately see return and we'll celebrate and we will do all these kind of press releases but the India will lose a competitive advantage on something which is very critical for the next decade."

ANI

18,753 Aufrufe • vor 5 Monaten

Elon Musk: As human intelligence plateaus, AI will scale to harness most of the power of the galaxy Elon: "I think we'll see intelligence continue to scale all the way up to where most of the power of the sun is harnessed for compute, and then ultimately most of the power of the galaxy." “ Once you think about artificial intelligence not as sort of this destination that you reach, but really as part of the overall escalation of intelligence, you know, human intelligence has also scaled as the population has increased, and we've been able to store more and more information.” “Because of population declines and low birthrate, human intelligence is somewhat plateauing and will actually decline.” “I think that we might have AI smarter than any single human at anything as soon as next year.” “And then probably within five (years), like say 2030, probably AI is smarter than the sum of all humans.” Friedberg: “ Do you think humans are on the decline because the AI is evolving?” “Do you think there's this evolution of the ecosystem on earth that's underway, that we don't really understand the structure of what's going on?” Elon: “ Yeah, maybe we implicitly know that it's coming.” Elon Musk david friedberg

Elon Musk: As human intelligence plateaus, AI will scale to harness most of the power of the galaxy Elon: "I think we'll see intelligence continue to scale all the way up to where most of the power of the sun is harnessed for compute, and then ultimately most of the power of the galaxy." “ Once you think about artificial intelligence not as sort of this destination that you reach, but really as part of the overall escalation of intelligence, you know, human intelligence has also scaled as the population has increased, and we've been able to store more and more information.” “Because of population declines and low birthrate, human intelligence is somewhat plateauing and will actually decline.” “I think that we might have AI smarter than any single human at anything as soon as next year.” “And then probably within five (years), like say 2030, probably AI is smarter than the sum of all humans.” Friedberg: “ Do you think humans are on the decline because the AI is evolving?” “Do you think there's this evolution of the ecosystem on earth that's underway, that we don't really understand the structure of what's going on?” Elon: “ Yeah, maybe we implicitly know that it's coming.” Elon Musk david friedberg

The All-In Podcast

107,012 Aufrufe • vor 10 Monaten

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

TBPN

85,358 Aufrufe • vor 5 Monaten

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Dwarkesh Patel

54,245 Aufrufe • vor 1 Monat

The fact that the universe doesn’t care about you doesn’t mean that other humans don’t care about you, or that we don’t have to care about other humans. That is, there is a purpose, there is a meaning: that is, to make people as well off as possible, to increase flourishing, to increase knowledge and life and health and freedom and safety. These are really, really meaningful. At least, I think they are. I don’t see anything wrong with them. I don’t think that that should leave you empty. And the fact that we live in a cold universe—that is, the laws of physics, the laws of biology, the behavior of viruses and bacteria and parasites and fungi—they don’t care about you. I’m sorry, they don’t care about you. And the sooner you realize that, the more you’ll be able to attend to the things that really do matter. Human Progress

The fact that the universe doesn’t care about you doesn’t mean that other humans don’t care about you, or that we don’t have to care about other humans. That is, there is a purpose, there is a meaning: that is, to make people as well off as possible, to increase flourishing, to increase knowledge and life and health and freedom and safety. These are really, really meaningful. At least, I think they are. I don’t see anything wrong with them. I don’t think that that should leave you empty. And the fact that we live in a cold universe—that is, the laws of physics, the laws of biology, the behavior of viruses and bacteria and parasites and fungi—they don’t care about you. I’m sorry, they don’t care about you. And the sooner you realize that, the more you’ll be able to attend to the things that really do matter. Human Progress

Steven Pinker

70,916 Aufrufe • vor 3 Monaten

In this scene from the 2016 movie Arrival, Louise learns that the non-human species perceives time non-linearly as a simultaneous "now" rather than a linear sequence of cause and effect. Contact with an intelligent non-human species that perceives time non-linearly would force humanity to rethink our assumptions about how time and space actually work. Physicists are already finding that the fundamental laws of quantum physics do not distinguish between past and future, suggesting that the forward flow of time we experience may not actually be a fundamental property of reality...

In this scene from the 2016 movie Arrival, Louise learns that the non-human species perceives time non-linearly as a simultaneous "now" rather than a linear sequence of cause and effect. Contact with an intelligent non-human species that perceives time non-linearly would force humanity to rethink our assumptions about how time and space actually work. Physicists are already finding that the fundamental laws of quantum physics do not distinguish between past and future, suggesting that the forward flow of time we experience may not actually be a fundamental property of reality...

Disclosure Party

1,452,970 Aufrufe • vor 6 Monaten

Q: "After doing AI for so long, what have you learned about humans?" Sam Altman: "I grew up implicitly thinking that intelligence was this, like really special human thing and kind of somewhat magical. And I now think that it's sort of a fundamental property of matter..." It's a fascinating thought. As we peel back layers of scientific discovery, we do seem to distance ourselves from the "main character energy" role in the universe. Yet, even if we aren't unique in our intelligence, there's a profound importance in our human existence that needs preserving.

Q: "After doing AI for so long, what have you learned about humans?" Sam Altman: "I grew up implicitly thinking that intelligence was this, like really special human thing and kind of somewhat magical. And I now think that it's sort of a fundamental property of matter..." It's a fascinating thought. As we peel back layers of scientific discovery, we do seem to distance ourselves from the "main character energy" role in the universe. Yet, even if we aren't unique in our intelligence, there's a profound importance in our human existence that needs preserving.

Bilawal Sidhu

1,183,430 Aufrufe • vor 3 Jahren

DAVID IBIYOMI with all due respect YOU ARE NOT GOD AND YOU DONT SC@R3 nobody It is really disappointing that someone tagged a man of God would say something like this that suggest using his influence to opprzz,pastor DAVID IBIYOME IS SAFE TO SAY YOU ARE NOT A CHRISTIAN,Jesus would not do this so you are not christ-like,well this post is to tell you that WE HAVE HAD ENOUGH OF THE LIKES OF YOU(ELITES) that uses the police and law enforcement to opprzz and this is to tell you that if you arrzzt anyone that didn’t do anything that warrants an arr3st we the RATEL MOVEMENT will get the person out,YOU ARE NOT GOD,YOU ARE HUMAN LIKE US

DAVID IBIYOMI with all due respect YOU ARE NOT GOD AND YOU DONT SC@R3 nobody It is really disappointing that someone tagged a man of God would say something like this that suggest using his influence to opprzz,pastor DAVID IBIYOME IS SAFE TO SAY YOU ARE NOT A CHRISTIAN,Jesus would not do this so you are not christ-like,well this post is to tell you that WE HAVE HAD ENOUGH OF THE LIKES OF YOU(ELITES) that uses the police and law enforcement to opprzz and this is to tell you that if you arrzzt anyone that didn’t do anything that warrants an arr3st we the RATEL MOVEMENT will get the person out,YOU ARE NOT GOD,YOU ARE HUMAN LIKE US

RAZOR BLADE

34,376 Aufrufe • vor 1 Jahr

We pre-train LLMs on the whole of the internet. You might think this explains how they learn so many emergent capabilities: the knowledge is implicit in the training data. But in fact models can do things that were never demonstrated anywhere in training! Sergey Levine argues that the real source of emergent capabilities is compositionality:

We pre-train LLMs on the whole of the internet. You might think this explains how they learn so many emergent capabilities: the knowledge is implicit in the training data. But in fact models can do things that were never demonstrated anywhere in training! Sergey Levine argues that the real source of emergent capabilities is compositionality:

Dwarkesh Patel

151,718 Aufrufe • vor 1 Monat

ELON: AI HAS DRAINED HUMAN KNOWLEDGE DRY - NOW DRINKING FROM ITS OWN WELL "You take the entire internet, all books ever written, all the interesting videos, and you distill that down into tokens, essentially bits of information. We've now exhausted all of that; the cumulative sum of human knowledge has been exhausted in AI training. That happened last year. So the only way to supplement that is with synthetic data, where the AI creates. It'll write an essay, or it'll come up with a thesis, and then it will grade itself. It'll go through this process of self-learning with synthetic data." Source: Elon Musk, Mark Penn, January 9th 2025

ELON: AI HAS DRAINED HUMAN KNOWLEDGE DRY - NOW DRINKING FROM ITS OWN WELL "You take the entire internet, all books ever written, all the interesting videos, and you distill that down into tokens, essentially bits of information. We've now exhausted all of that; the cumulative sum of human knowledge has been exhausted in AI training. That happened last year. So the only way to supplement that is with synthetic data, where the AI creates. It'll write an essay, or it'll come up with a thesis, and then it will grade itself. It'll go through this process of self-learning with synthetic data." Source: Elon Musk, Mark Penn, January 9th 2025

Mario Nawfal

1,938,755 Aufrufe • vor 9 Monaten

🚨 FRANCOIS CHOLLET SAYS THE CURRENT LLM STACK IS THE WRONG PATH TO AGI Interviewer: “When will we accomplish the first definition of AGI?” Chollet confirms we are already on that exact trajectory, pointing out that: "Current technology can fully automate at human level or beyond any domain where you have verifiable rewards, right? And code... code being the first one." He then brings up the problem with LLMs, stating that: "It's possible in principle to build something that looks a lot like AGI on top of the LLM stack... I do believe, however, this would be the wrong thing to do because it would be very inefficient." Looking ahead to what the final architecture will actually require, he explains that: "AI research will have to trend towards not just efficiency, but in fact optimality over time. And for this reason, future AI in a few decades... it's not going to be this harness on top of a reasoning model on top of a base LLM."

🚨 FRANCOIS CHOLLET SAYS THE CURRENT LLM STACK IS THE WRONG PATH TO AGI Interviewer: “When will we accomplish the first definition of AGI?” Chollet confirms we are already on that exact trajectory, pointing out that: "Current technology can fully automate at human level or beyond any domain where you have verifiable rewards, right? And code... code being the first one." He then brings up the problem with LLMs, stating that: "It's possible in principle to build something that looks a lot like AGI on top of the LLM stack... I do believe, however, this would be the wrong thing to do because it would be very inefficient." Looking ahead to what the final architecture will actually require, he explains that: "AI research will have to trend towards not just efficiency, but in fact optimality over time. And for this reason, future AI in a few decades... it's not going to be this harness on top of a reasoning model on top of a base LLM."

Chris

15,578 Aufrufe • vor 3 Monaten