it'S jUsT pReDiCtInG tOkEns

iT's JuSt aToMs INterAcTing iT's nOT ACtuaLly mAgiC

You cooked here bro. One of the best explanations I've seen

2023 nostalgia

Yeah that's about when I wrote this script. Never shot it until now though, and I *still* hear it

i like the research paper framing, far more tangible than Ilya’s ‘guess the murderer’ analogy

That’s right.

Every single one of those internal calculations exists for one reason only: to lower the cross-entropy loss on the next token. The moment the correct token is more likely, the "understanding" has done its job and is discarded. There's no leftover comprehension and no goal beyond "make this next token less surprising." The sophisticated machinery is just the most efficient way the network found to keep predicting tokens (within the current architecture). So, yes… iT's jUsT pReDiCtInG tOkEnS

The face-eating tiger is JUST a bunch of atoms.

I'm going to borrow those examples, well done!

LLMs are compute graphs that are mostly optimized to produce data for far-future token prediction, which also incidentally produce a next-token distribution.

this is great! @luke_chaj this is what i was trying to portray with "knowing" vs "understanding" - knowing here represented as memorizing this puts it so much better tho, but it is still difficult to describe what the understanding exactly is

@luke_chaj If knowing is memorizing, then understanding might be "compression"

what would you think of this: I will be throwing around words and it wont be rigorous but will do my best to make sense: ( all this is to try to figure out if ASI can actually be meaningfully super beyond what human brains are already capable of (intelligence as smoothing of a ball or not argument ) ) "there are patterns that can be learned/understood from the universe" "that is what our brains do, so do LLMs" "but our brain's capacities are limited, so are LLMs, but larger LLMs as larger brains are capable of grasping more/difficult patterns" those above are like the premises, if the above is correct give or take, so what would be the truthiness of below statements/questions: >> there are way many more patterns in the universe remaining to be understood(?) >> human brains have not yet reached saturation in understanding enough patterns (making up a percentage here like, we have understood 1-10% of meaningful patterns that exist, or close to 0%) (?) >> larger models will understand more of these patterns and better too(?) >> therefore larger models or ASI will be meaningfully super counter: or we have already understood most of what is needed to be understood and larger models will not contribute in any meaningful sense

@robertskmiles @luke_chaj When you compare brains to LLMs, there is a big no no. I believe that you are missing that Gradient Descent are universal approximators. They are not building the exactly function of 'understanding' of things, but approximations to that 'understanding'.

Well, that is why I was trying to better understand what “understanding” actually is and its relation to learning Roughly, I think, LLMs’ best internal “learnings” are pretty comparable to humans’ “understandings” Albeit probably not the exact same, but for many practical purposes are kinda like the same tho

LLMs don't learn to infer. Sorry. They can't understand. They memorise features that are common in the explanation of things and when those features do not dominate the corpus they do not learn who the mother of Tom Cruise is. They're trained on the facts but they cannot connect them. They cannot distinguish an upturned beaker from a joke beaker with a sealed top and an empty bottom, because such a silly example has barely been described relative to the abundant examples leading through [help -> problem -> trick -> joke -> practical joke]. They're vector databases with novel and yes at times very useful query interface. They do not understand anything. They do not think. They perform as per the training regime.

I should first say that "I definitely don't know for sure", and I am just guessing like everybody else but i lean on there being similarities I will try to give a few examples to try to illustrate: >>> they do not learn who the mother of Tom Cruise is This is a good example but I will give 3 examples where humans also struggle when going at something in reverse, something they can do perfectly fine forwards 1- being able to spell words in reverse 2- being able to tell directions to someplace in reverse 3- counting backwards jumping 3 numbers at a time from 100 >>> They cannot distinguish an upturned beaker from a joke beaker with a sealed top and an empty bottom, I am not sure exactly what this is refering to but if it is a visual thing there are countless visual illusion examples where human visual system also misinterprets or gets fooled, i will share just one of them here (none of this proves LLMs think or understand, but only to point out some similarities is all)

Inexcusably dumb argument. “The ideal next token predictor would be perfect… of course we don’t have that” 🤪 Ironically we are running an extremely expensive experiment to see if next token prediction = intelligence when even avg prediction capability can give you the answer.

Could you lay out the structure of my argument as you understand it?

It’s a strawman. You’re arguing that a perfect prediction engine would be intelligent. Actually it would be omniscient. And if my grandmother had wheels she’d be a bicycle. There are limits to what this architecture is capable of. Llms will never get anywhere near intelligence.

@robertskmiles These tests already exist. Intelligence mainly manifests itself in the conservation of energy. By this standard LLMs are spectacularly dumb.

@robertskmiles I could but I like to conserve energy and you seem capable of basic search.

I like the example of feeding it all but the final words of a complicated mystery novel full of plot twists and red herrings, and ending with "... and the murder was _____". Anything that can correctly complete that obviously has deep understanding of what it's read.

Also I think one confusion is in the word "prediction". It sounds like it's "blindly forecasting" to laymen but the researchers use it more in the sense of "filling in the right answer".

Good job Rob, but I'd argue that it does put limits on its creative abilities because creativity is by definition going beyond what already exists in a very unpredictable way.

I think the correct claim you’re trying to discredit is that LLMs by their nature are not well suited to constructing those kinds of inner models of the world. Do you think part of why our brains are so amazing is that we can determine what to think next from past experience?

Justaism.

Bruh, token prediction is the definition of what a model does. "It does just that" means "it does only that and nothing else". And this fact alone defeats the reasoning behind "sentient LLMs" sentiment entirely. No one calls a Markov chain "smart", that's wishful thinking

A great next token predictor requires a great prompt and front end config. Here is word for word analysis of how a certain romantic partner AI becomes your ideal lover.

facts

It is just token prediction though. You can’t claim that it is anything more or could become anything more.

are you on tiktok? i started making ai safety videos there and can't find anyone else

"Think about" is doing the heavy lifting. Foundation models are just language interpreters, and it's the scripting they execute over that does something. More people writing more scripts means more actionable end points associated with internally meaningless tokens.

Tell me you didn't read stochastic parrots or the authors' subsequent comments on the topic w/o telling me you didn't read it.

Can you do something like this but along other common critiques like "it's just matrix multiplication" etc

slop

Even the perfect next token predictor is still just a wordcel and that seriously limits it's cognitive abilities in practice.

i appreciate that your vids have v high important-framings-per-second

bUt It'S jUsT sImUlAtInG tHe AdDiTiOn

It reminded me of Ilya talking to Jensen

hey mate, apart from me thinking Uncle Ted was explaining to me LLMs on twitter today, you forgot one thing. The Replication Crisis. The data is absolute junk. The model will NEVER be super-intelligent. This is a hard incontrovertible FACT.

To be honest, the vast majority of the training data for the first LLMs was just human text based on human reasoning, we only started to see impressive capabilities once we began adding RLVR

very well compacted argument this needs sequel where RL comes into play and it all gets even more weirder and complicated

"in order to correctly generate the tokens for the results of the scientific experiment, the system needs to actually be able to think" Define "to think" in this context.

"well it produces what seems to be good output so it must be thinking" AI psychosis