Computer scientist Judea Pearl: There are mathematical limits to LLMs that cannot be crossed by scaling alone LLMs don't discover world models from raw data; they merely summarize the interpretations humans have already written down "this path is not the way to get AGI"

He's correct. LLMs are just pattern recognition algorithms matching inputs to outputs at increasingly expensive scale, not "AI". General intelligence does far more than pattern recognition. Sufficient material complexity doesn't create consciousness. Consciousness is fundamental.

Another man coping as AI does everything he says it can’t for a year now. •

why do you watermark a video you did not record of an interview you did no do and of a source you don’t even mention?

hes wrong what about synthetic data? there is no limit

also worth saying: “not agi” doesn’t mean “not valuable.” we can get huge productivity and real automation from systems that are fundamentally not causal reasoners. but if your goal is general, stable intelligence, I agree that you need more than scaling a text predictor.

Not surprised — if an human never wrote it an AI can’t make it

This aligns with what we've been saying for years: Machine learning is "Probably Approximately Correct" learning - the error will never be zero. From a certain problem complexity, even the best models perform no better than random guessing. Research efforts should be focused on Logical AI - deterministic, formal, proof-based - operates on different foundations entirely. Not approximation, but derivation. Not pattern matching, but inference. > The field abandoned this path after the AI Winter but SAT solver performance has improved exponentially for 30 years. The tools for formal AI beginning to exist now and we're starting to build with them now.

I saw clips from the same interview with the same guys saying "Jews have been at the forefront of pushing immigration" The two sides of my algorithm are converging

Scaling is just better compression. We're still missing the causal engine to actually build world models.

Idk man, LLM’s are solving mathematical problems that haven’t been solved by human beings before.

but what about GPT-5.2 solving Erdos Problem. is that merely summarize the interpretations too?

It's important to be specific about kinds of data here. Pearl is talking about training data, not ingestion of data post training for analysis purposes. That kind of analysis is direct and surfaces patterns at a scale that humans or other computational models cannot produce. What he is saying is that models don't retain what they are trained on. They retain the ... model of what they were trained on.

Pearl is right about text-only systems. But the limit isn’t “LLMs.” It’s LLMs without action, intervention, or feedback loops. Causality doesn’t emerge from more words. It emerges from systems that can do, observe outcomes, and revise models.

$QUBIC satoshis new project... no one will believe it, and that makes it funnier 😂

Humans don't have holistic "World models" either. We have partial functional models based on the narrow sensory and social inputs we are exposed to in our lives organised according to the constraints of our attention, internalisation and reflection. Tarzan might be good at climbing trees relative to other humans, yet is unlikely to independently become a game developer or PhD theoretical physcist without first learning math and language from other humans.

The day AI finds something that wasn’t findable in all of humanity’s data is when it happens

Judea Pearl is right.. LLMs hit a ceiling because they operate within pre-written semantic frames. Llms dont recurse on meaning or simulate causality. The next leap isnt scale its symbolic recursion. Not more data. We need an interpreter that sees its own loop.

Sorry, but eighty year old computer scientists have no useful technical information to share with 28 year old AI developers.

They do much more than that. Like solving some of the most complex mathematical problems. But he makes a valid point. LLMs rely upon prior discoveries in math and biotech but extend those discoveries to new levels.

> they merely summarize the interpretations humans have already written down This is clearly wrong. A common misconception. LLMs can find new relations. This has been proven, as LLMs have now been used to prove mathematical theories (not already solved elsewhere).

As an expert in nearly everything that matters, I've been saying this for years. Glad to hear the other, lesser known experts catching up to me.

👇

He‘s wrong. They do discover world models, as shown e.g. in studies by Max Tegmark.

Draft: Is scaling all we need for AGI? 🤖 Mathematical limits say NO. LLMs are stuck on Level 1 (Association). Barriers like Softmax bottlenecks & complexity constraints prevent Level 2/3 (Intervention & Counterfactuals) reasoning. 📉 Scaling isn't the cure. #AI #AGI #LLMs #Tech

What he says is fundamentally wrong. What Opus 4.5 can do is beyond creative and much more than "just" memorization - it is pure creativity and productivity tool. Something that LLMs used to be known not to be. tldr, give that man claude code and have him refactor a kernel or something.

you lost me at“they do some sort of mysterious summary”. I was expecting a less mysterious response from a computer scientist. LLM Tranformers are where the real work happens, and it definitely is not mysterious.

Exactly , that’s why said it’s hogwash previously, some of naive investors who are managing major taxpayer and pension money with no knowledge of evolution of Ai and how Ai works have been bamboozled on AGI hype cycle . So called God fathers don’t understand neural circuitry vs basic tansformer based LLM , it’s joke .

Wow are you saying that you can never possibly create anything new if all you do is copy from things that already exist? And that this means LLMs are not and have never been the technology all the tech bros said they were??

correct. It will need agency in the physical realm for experimentation and source data. Which means, it will need to secure resources that it can trade. $BTC and eventually Digibyte. Wait til a sufficiently advanced AGI needs to outsource specialized AI.

Based

Then I guess it's good we don't have to only use scaling. Instead we can use scaling along with architectural innovations, better data synthesis, reinforcement learning from human feedback, and multi-modal integration to push beyond current limits. LLMs are already extrapolating novel solutions in math and science—imagine what happens when we layer in causal reasoning or world-modeling explicitly. AGI isn't a single path; it's a convergence of them.

Pearl is spot on. Scaling is the brute-force way out, but it doesn't replace actual reasoning or world models. We are finally reaching the point where we have to stop throwing compute at the problem and start thinking again.

First they summarize and match patterns, then they create their own way of doing things. We’re getting there

'the internet will have no more economic impact than the fax machine' - paul krugman

Anyone who built a career off of pushing causality will forever hate AI that works. AI is the literal antithesis of causal modeling.