Nando de Freitas's banner

Nando de Freitas

@NandoDF • 110,513 subscribers

I seek to understand intelligence & agency and build AI aligned with compassion, freedom & universal human empowerment through progress in science & engineering

Shorts

Predicting the next word "only" is sufficient for language models to learn a large body of knowledge that enables then to code, answer questions, understand many topics, chat, and so on. This is clear to many researchers now, and there are nice tutorials on why this works by Ilya Sutskever resorting to compression ( ) and by Geoffrey Hinton ( ). However, the emergence of types of understanding is not unique to language models. In by Misha Denil and Brandon Amos the authors trained models to predict the next few time stems of over a hundred robot hand sensors (Touch, Gyro, Accelerometer, Joint Info, Actuator Info, etc.). They ten found out that they could regress the shape of the thing the hand was touching from the activations of the neural networks using probes. That is, the model developed an internal representation of shapes even though it was simply used to predict "only" the next few senses. Awareness follows from simple predictions and interaction with the world.

Predicting the next word "only" is sufficient for language models to learn a large body of knowledge that enables then to code, answer questions, understand many topics, chat, and so on. This is clear to many researchers now, and there are nice tutorials on why this works by Ilya Sutskever resorting to compression ( ) and by Geoffrey Hinton ( ). However, the emergence of types of understanding is not unique to language models. In by Misha Denil and Brandon Amos the authors trained models to predict the next few time stems of over a hundred robot hand sensors (Touch, Gyro, Accelerometer, Joint Info, Actuator Info, etc.). They ten found out that they could regress the shape of the thing the hand was touching from the activations of the neural networks using probes. That is, the model developed an internal representation of shapes even though it was simply used to predict "only" the next few senses. Awareness follows from simple predictions and interaction with the world.

134,406 次观看

Trading and investment conference, or AI conference? Have a guess 🙃

Trading and investment conference, or AI conference? Have a guess 🙃

57,807 次观看

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

This was an amazing week at ⁦Microsoft AI⁩ !! We released MAI 1 preview and a taste of MAI Voice. I’m super happy with this team - only about 100 people and already shipping in ⁦Arena⁩ in less than a year. Strong support. More soon. Thanks for feedback!

This was an amazing week at ⁦Microsoft AI⁩ !! We released MAI 1 preview and a taste of MAI Voice. I’m super happy with this team - only about 100 people and already shipping in ⁦Arena⁩ in less than a year. Strong support. More soon. Thanks for feedback!

Nando de Freitas

50,245 次观看 • 1 年前

Google Waymo is an unbelievable AI product. After a ride on a self-driving “taxi” in SF, I felt truly emotional about how far the field had advanced. Definitely feeling the AI.

Google Waymo is an unbelievable AI product. After a ride on a self-driving “taxi” in SF, I felt truly emotional about how far the field had advanced. Definitely feeling the AI.

Nando de Freitas

44,543 次观看 • 2 年前

没有更多内容可加载