How does Ilya have such good research taste?

Full interview here:

I am curios to see how SSI plays out

it's very rare for someone to have this high IQ and have good taste + be very good at explaining as well absolutely brilliant

i watched this podcast up until two times now to actually understand every bit of it (1st time was a skim through), so the key to building a taste is having an approach in a simpler way & the correct way with beauty of design in mind and that;s how it should be to build a top-down belief.

imo research taste is mostly pattern matching across domains, not some mystical filter. ilya saw search+value functions work in alpha go and bet it transfers to language. saw scaling work in vision and bet text follows. that takes conviction when evidence is thin. most people confuse recency bias with taste

Uses (@scholarxiv)

Finally coming back to Spiking Neural Networks

🐐🐐🐐

honestly i think it just comes from years of doing the work and learning what actually matters

can we have you, Ilya, and @RickRubin on one episode? trust me I have good taste. This will be a hit

Interesting that he mentions local learning rules. Backpropagation definitely isn't local.

Going to fundamentals and thinking about something always results in some real-world changing ideas.

Post-Soviet Union roots. Blindly trusting in superficial metrics is a very much American /Western thing.

I love how he emphasizes going back to fundamentals, the first principles. Such a reductionist (in the best way) and elegant way to approach a problem.

"Brain has these organs. Folds probably don't matter. Why do you want the neuron? Because there's many of them and kinda feels right." Can't make this s* up! Great parody! 😂🤣

Knowledge is making a comeback apparently.

he did such an amazing job of answering that impossible question some things cannot be replicated or explained.

He needs a hair transplant

resonator

He's 100% spot on and - without boasting about it - I have to be frank and say I approach research like this too. It's the only way to solve abstract topics

vibe coding vibe coding?

'An aesthetic of how AI should be by thinking about how people are' - God, day 4

And not just research taste. Look at the man’s T-shirts🤩

Sutskever hire me:(

Ilya has always had great intuition in AI. We want him back in the arena and show us his new inventions.

Probably the same way anyone does. He spent years actually building things, failing, and recalibrating. Taste is just pattern recognition with scar tissue.

Yes, it's elegance and beauty, simplicity and purity help define taste.

Subtle meaning of the image on Ilya’s t-shirt

because he is a pro

Why everyone talks about Ilya and no one talks about Alex?

You need to have very good priors

When will you do a followup?

Was thinking about how the foods could matter, maybe design the NN to mimic the brain more closely but then why not copy the brain as closely as possible. Things are going to get real weird when AI takes the form of lab grown brains

he was doing Alexnet before it was cool