Loading video...
Video Failed to Load
Terence Tao says that we still cannot predict how many weights a model needs, or how long training takes, to model a data set well. It is done empirically, with rules of thumb, because some very basic concepts are missing. Communication has Shannon's theory: information comes in bits, with... show more
14,653 views • 5 days ago •via X (Twitter)
7 Comments

I mean... he had a point. The scaling law was largely empirical or with limited rigor in mathematical evidence.

Tao is a know nothing hack and you’re a fucking idiot for giving his noise more attention and respect than it deserves. He’s showing why it’s not Asian dudes doing math that actually advance the world but white men, and logicians like me who can prove he knows nothing. Fuck him.

@burny_tech He’s an imbecile if he doesn’t know who Yann Lecun is, or has never used an open weight model.

"With four parameters I can fit an elephant, and with five I can make him wiggle his trunk."-- John von Neumann

Absolutely no reason to believe this

Isn’t that what validation is for. It helps you gauge accuracy.

i. There shouldn't be "AI" when there is already tech and ML. There shouldn't be "AI" at all. The limits of "AI" lie in its lack of definitions, its redundancy, its lack of ethics, and its unlawfulness. ii. When it comes to ML, correct, these are mostly questions empirical in 1/
