Nando de Freitas's banner
Nando de Freitas's profile picture

Nando de Freitas

@NandoDF • 110,513 subscribers

I seek to understand intelligence & agency and build AI aligned with compassion, freedom & universal human empowerment through progress in science & engineering

Shorts

Predicting the next word "only" is sufficient for language models to learn a large body of knowledge that enables then to code, answer questions, understand many topics, chat, and so on. This is clear to many researchers now, and there are nice tutorials on why this works by Ilya Sutskever resorting to compression ( ) and by Geoffrey Hinton ( ). However, the emergence of types of understanding is not unique to language models. In by Misha Denil and Brandon Amos the authors trained models to predict the next few time stems of over a hundred robot hand sensors (Touch, Gyro, Accelerometer, Joint Info, Actuator Info, etc.). They ten found out that they could regress the shape of the thing the hand was touching from the activations of the neural networks using probes. That is, the model developed an internal representation of shapes even though it was simply used to predict "only" the next few senses. Awareness follows from simple predictions and interaction with the world.

Predicting the next word "only" is sufficient for language models to learn a large body of knowledge that enables then to code, answer questions, understand many topics, chat, and so on. This is clear to many researchers now, and there are nice tutorials on why this works by Ilya Sutskever resorting to compression ( ) and by Geoffrey Hinton ( ). However, the emergence of types of understanding is not unique to language models. In by Misha Denil and Brandon Amos the authors trained models to predict the next few time stems of over a hundred robot hand sensors (Touch, Gyro, Accelerometer, Joint Info, Actuator Info, etc.). They ten found out that they could regress the shape of the thing the hand was touching from the activations of the neural networks using probes. That is, the model developed an internal representation of shapes even though it was simply used to predict "only" the next few senses. Awareness follows from simple predictions and interaction with the world.

134,406 views

Trading and investment conference, or AI conference? Have a guess 🙃

Trading and investment conference, or AI conference? Have a guess 🙃

57,807 views

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

This was an amazing week at ⁦Microsoft AI⁩ !! We released MAI 1 preview and a taste of MAI Voice. I’m super happy with this team - only about 100 people and already shipping in ⁦Arena⁩ in less than a year. Strong support. More soon. Thanks for feedback!
2:05
NandoDF's profile picture

This was an amazing week at ⁦Microsoft AI⁩ !! We released MAI 1 preview and a taste of MAI Voice. I’m super happy with this team - only about 100 people and already shipping in ⁦Arena⁩ in less than a year. Strong support. More soon. Thanks for feedback!

Nando de Freitas

50,245 views • 1 year ago

Google Waymo is an unbelievable AI product. After a ride on a self-driving “taxi” in SF, I felt truly emotional about how far the field had advanced. Definitely feeling the AI.
0:26
NandoDF's profile picture

Google Waymo is an unbelievable AI product. After a ride on a self-driving “taxi” in SF, I felt truly emotional about how far the field had advanced. Definitely feeling the AI.

Nando de Freitas

44,543 views • 2 years ago

No more content to load