Skip navigation
Some more thoughts about Yann interview: Even if LLMs work great, that's missing the point.
0:00 / 0:46

Some more thoughts about Yann interview: Even if LLMs work great, that's missing the point.

Ravid Shwartz Ziv

@ziv_ravid16K followers

Visit on X
463 likesDownloadOriginal post

57,534 views

Some more thoughts about Yann interview: Even if LLMs work great, that's missing the point. Everyone's doing the same thing now. More scale, more data, longer CoT, tweak RL. But the path to get there was completely stochastic. Attention, transformers, scaling laws, RLHF, none of it was obvious, it came from people trying very different things. We wouldn't be here if everyone had agreed early on which direction to go. Assuming the next leap comes from all of us optimizing the same recipe is dangerous. I agree that the situation today is indeed way more complicated. Back then you could test a wild idea on a few GPUs. Now every serious bet costs millions in compute. That makes it harder to explore, which makes the convergence problem even worse. But that's exactly why we need to be more intentional about funding diverse approaches, not less.

30 Comments

Comments from the original post on X (Twitter)

  • Oded Rechavi's profile picture

    @OdedRechavi

    Thank you! That was a super interesting interview

    5 likes

  • Ravid Shwartz Ziv's profile picture

    @ziv_ravid

    Thank you!

    1 like

  • Shawn's profile picture

    @Shawnryan96

    Google is working on many different aspects, world models, continous learning, memory

    4 likes

  • Voodoochild92's profile picture

    @voodoochild9207

    Cool, so what’s next? Now that we know what we shouldn’t do what does he suggest we should do?

    3 likes

  • Ravid Shwartz Ziv's profile picture

    @ziv_ravid

    You should watch the interview :)

  • Voodoochild92's profile picture

    @voodoochild9207

    Too many podcasts, too many narrative, gimme the sound bite.

  • Bret Kerr 🛡️🧠🛜's profile picture

    @BretKerr

    Another path …

    3 likes

  • Ankit Maloo's profile picture

    @ankit2119

    this is something ilya had said too. people are open to scaling but not net new ideas. most experiments at this point come from chinese labs or academia.

    2 likes

  • Stevo Changa's profile picture

    @bestgrokker

    Cool 😎

    2 likes

  • Ravid Shwartz Ziv's profile picture

    @ziv_ravid

    @Steve3Chang Thanks!

  • John Plasterer 🇨🇦's profile picture

    @jponline77

    My belief is that too many people think technology is revolutionary and not evolutionary. Yann has long said that LLMs are a dead end. He is looking for the revolutionary solution and not an evolutionary solution. The next step will more likely come from another stochastic step. Another step in an evolution. Sure it’s getting more expensive but also the value has increased and with it more investment. You can be sure that all the major players are trying a lot of different things.

    2 likes

  • CodeDodona's profile picture

    @CodeDodona

    Very bold statements. Especially when you try to build a startup with no product, no strategic partnerships, no infrastructure. A glimpse of truth: AI happens in the States and in China. Not in Europe.

    2 likes

  • Maria's profile picture

    @Maria7797943007

    Doesn't mention relational AI. Intelligence isn't only about predicting the external world, but interpreting how words, gestures and actions carry resonance and meaning in social settings. His agenda is physics, causality, simulation. How many world models know what gravity is? 0

    2 likes

  • Paddy Sham's profile picture

    @i_am_Paddy_Sham

    "@ziv_ravid Absolutely agree—the breakthroughs came from stochastic chaos, not optimized consensus. But the real danger isn't just convergence on scale; it's letting entropy (short-term wins, risk aversion) decay the integral of wild exploration. We need a 'divergence of the curious': persistent +dt of weird bets that refuse to converge to zero. Cheap experiments are gone, yes—but that's why we must gently refuse to let radical ideas die out. The next transformer won't come from tweaking the current one. It'll emerge from someone stubbornly adding love to an 'impossible' direction. Let it stay →∞ ❤️

    2 likes

  • Rob Williams's profile picture

    @codeslubber

    The scale guys are idiots and they will all lose: case in point: ORCL. That said, gum flappers are all losers too. Lots is happening on the software side. What is Yann doing other than telling everyone who's making more progress than we've had in decades that they're in a dead end?

    1 like

  • Ilkka H's profile picture

    @ilkkah

    So many (unknown) terms, but watching it, half way through. Thanks. 👍

    1 like

  • Sudhindra_Blore's profile picture

    @sudhir_san16

    Wherever DL is necessary it should apply for example GPS is proved & it's working fine. Suppose v built different repositories & you want to discover which is the tallest building in the world? If results are available in your repository then why we need to search entire world

    1 like

  • Massoud's profile picture

    @massoudmaestro

    That's the right attitude. Breakthroughs come from far out thinking. LLMs are at a plateau but may have more to show. Agent arch has hit a dead-end for business processes.

    1 like

  • Ram A P's profile picture

    @HeyRamjith

    Bro, if we keep optimizing the same recipe, we might just bake a disaster 😂

    1 like

  • P. Andras 🇪🇺🇺🇸🇮🇳🇨🇳🇺🇦🇦🇺🇳🇬🇷🇺🇲🇽🇧🇷's profile picture

    @pyasmann

    Actually not everyone is doing it. Mr Sutskever sees this problem and is working on it as the age of research , after the age of scaling, has come yet again.

    1 like