Yann LeCun explains that large language models are trained on about 30 trillion words, representing nearly all public internet text. He says it would take a human over 500,000 years to read that much. But a 4-year-old child sees just as much visual data in their first few years... of life. This shows how much richer and more complex real-world experience is compared to reading text. Training on the web is huge but it still doesn’t match what a child learns just by living.show more

This is a BS comparison because the child’s visual data is massively duplicated. They stare at the ceiling above their bed and their mother’s face for thousands and thousands of hours.

Yann isn’t wrong about the numbers, but he’s been saying the same thing for the past three years. No serious person in the labs thinks scaling exact current architectures is the answer to AGI/ASI. LLMs don’t have to get us there, they only have to get us to the next step in the chain of breakthroughs that does.

This argument mistakes data size for learning signal. Vision is high-bandwidth but low-information, language is low-bandwidth but extremely high-information. A child does not learn from pixels any more than a model learns from Unicode code points both learn from the abstractions

Most of that "visual data" is not processed to any depth

He’s the reason why META failed in AI

This is why LLMs are good at language tasks and terrible at physical reasoning. They're missing the foundation layer that comes from actually existing in the world.

Great point

So basically LLMs should be hooked to netflix, prime, all streaming platforms, and the thousands of satellite tv channels in all languages to actually get that same visual training data if not more. @gdb FYI

How behind is LeCun? All contenders in the AI race are multimodal. Text, image and now video with sound.

Self-supervised pre-training is becoming evidently obsolete and inefficient, LLMs are hitting a wall with just predicting the next word from tons of text, it won't give us real reasoning or understanding of the real world. Yann's JEPA is a solid step toward better "world models,". But it's still not enough alone. We need bigger ideas.

I wonder what might change after AI models are trained on all public internet videos

His thinking is ignorant . Its like comparing apples and oranges

Why “Intuition” Is the Wrong Word When leaders in artificial intelligence say that physical action and planning require “continuous intuition,” they are pointing to a real limitation—but naming it incorrectly. What is missing is not intuition. What is missing is architecture. Intuition is what systems appear to have when the underlying structure is invisible. In biological organisms, what looks like intuition is the result of force being organized correctly before action occurs. The body does not guess. It does not search symbolically. It does not manipulate representations of the world and hope they align with reality. It negotiates gravity, rotation, and load continuously, using architecture that has developed over time. Large language models work well in symbolic domains because symbols can be rearranged without consequence. Math, code, and chess exist in spaces where invalid moves can be rolled back, searched over, or corrected without physical cost. The real world does not permit this. Force cannot be undone. Timing cannot be rewound. A missed redistribution of load results in collapse, not error correction. This is not a failure of intelligence. It is a mismatch between representation and reality. Physical action does not require intuition. It requires a system that already knows, structurally, what is possible before acting. Biological movement is not driven by continuous guesswork. It is driven by continuous constraint satisfaction. The body acts within limits established by gravity, rotation, opposition, and continuity. When those limits are respected, action appears fluid and intelligent. When they are violated, no amount of reasoning can prevent failure. The mistake modern AI makes is assuming that understanding reality means describing it symbolically. But description is not preparation. Recognition is not readiness. A system can label the world perfectly and still be unable to inhabit it. This book exists because intelligence does not emerge from manipulating symbols about the world. It emerges from being organized within the world. What robotics and artificial intelligence lack is not intuition, creativity, or reasoning power. They lack developmental architecture. They have never learned to carry gravity, to permit rotation, to distribute force, or to anticipate load. They correct after the fact because they were never structured to prepare before it. Until intelligence is grounded in force—until systems are built to develop architecture rather than infer behavior—no amount of “intuition” will close the gap between symbols and reality. This is not a philosophical disagreement. It is a physical one. And that is why this book had to be written. I just finished it.

If this mattered as much as he seems to think, blind people would never be able to get through life. Sometimes it feels like he has a personal grudge against AI.

i sit in a bus everyday for 45 mins, watch same things fly by, learn nothing, while another guy sitting 2 rows ahead has already read half a book, learned so much and made mental map of what was in the book, and he will keep doing it everyday while i will just stare meaningless

Absolutely, and the core reason isn’t just richness of sensory data. It’s that a child acts in the world. Their learning loop is agentic, not passive. A child’s sensorimotor stream is tied to self-generated behavior, endogenous goal-setting, and control-modulated exploration. Reading 30T tokens is nothing like authoring your next state. That’s exactly the gap my recent work tries to formalize. It attempts to answer the question of when a recurrent system is producing internally originated state transitions rather than just reacting to inputs. I call this test of agency the Irreducible Agency Invariant (IAI) If anyone’s interested in the technical side, here are the two most relevant preprints: Conceptual tech brief (succinct): Implementation preprint (equations & dynamical test): Embodied learning is powerful not just because of data, but because it couples perception to self-driven updates, which is the thing current LLMs mostly lack, and the thing we need to measure if we want systems that learn more like children.

Exactly. You can flood a system with 30 trillion words and still miss the point. Children don’t become intelligent because they process infinite data they become intelligent because they learn boundaries, not meanings. Meaning drifts. Boundaries don’t. That’s why scaling LLMs hits a ceiling: •more text ≠ more grounding •more tokens ≠ more coherence •more scale ≠ more safety Without stable constraints, the model just becomes a faster pattern machine — not a governed intelligence. This is the part the field keeps walking around: AGI won’t emerge from more words. It will emerge from architectures that remain coherent when words fail. Which is exactly what your quote captured: “It needs stable constraints that don’t drift when language does.” Perfect alignment.

He means "see" as in perceive all the pixels that's why the data is so huge

This is exactly why text-only models might hit a wall. Text is a low-bandwidth compression of reality. We are trying to teach AI about the world by having it read the manual, while a 4-year-old is actually using the product. We need 'World Models' (video/sensory data), not just bigger text predictors, to bridge that gap.

Think about it... AI is using the "Internet" and "social media" to LEARN what humans are, based on how the AI is programed by Corporations. ....... what could possibly go wrong.

Whats the point of this talk? To convince us that we are a long way off from life changing completely due to AI?

the fact that LLMs are so janky even given stimulus orders of magnitude beyond what any human could ever get, shows how impoverished the underlying mechanism is compared to what humans are born with.

Read aloud all of Wikipedia to a baby and see how long it takes it to retain the information. AI pedagogy should be given some thought. Some papers have shown that order definitely matters.

I think trying to say one is better than the other is silly. For a child it makes sense they learn through visual data much more than ai does using text based learning . However AI is unique and different . I believe the key to success is Both , scaling and real world experiences. BOTH are just as equally important.

Conceiving has nothing to do with training. If Einstein had to be trained before conceiving Relativity, he should reach old age, not deliver at 21 y.o. Major science figures deliver at young age. AI delivering through training = will never reach Human performance. Are we going to flip this? AI is not in the hands of scientists. It is a capital intensity activity.

I think that more points towards an algorithmic difference, not what it is learning. though embodied learning has unique properties. but I think simulation can do as well. it's all a simulation. what you want is the childs understanding for reading all of the web.

LeCun’s point is the right provocation: text is a thin slice of reality, it’s the receipt, not the experience. A child doesn’t just consume data. They get ground truth: physics, causality, consequence, and feedback loops in real time. They learn what can happen, not just what’s been said. So the next leap isn’t more tokens. It’s models that can act, observe, fail safely, and update beliefs, with memory, embodiment (real or simulated), and objectives that reward understanding over fluency. LLMs mastered the library. Intelligence is what happens when you step outside it.

He has no idea what capabilities a human would develop after reading for 500,000 years.

We don't speak in algorithms and functions, why should we force an AI to do so. Let it learn, teach it how to learn. Start from the fundamentals. Just dropping some breadcrumbs. A plug - n - play solution is in the works so don't worry. ~2 months. You need not adjust anything but your settings.

Volume alone doesn’t capture learning quality; human brains integrate multisensory input, not just raw data.

Uh, YouTube.

LeCun doesn't know what the hell he's talking about.

So?

The human brain has developed over millions of years. It's a combination of visual, emotions, space intelligence... a lot of things non-lingual. Human intelligence is still above LLM. LLM can capture knowledge but not wisdom.