Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

.Reiner Pope works out from first principles how much frontier models are overtrained relative to Chinchilla optimal. One of the cleverest deductions from the lecture, and the one I enjoyed the most.

dwarkesh_sp's profile picture

Dwarkesh Patel

243,524 subscribers

55,007 görüntüleme • 3 ay önce •via X (Twitter)

EğitimHaberler & PolitikaBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Did a very different format with Reiner Pope – a blackboard lecture where he walks through how frontier LLMs are trained and served. It's shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a bit technical, but I encourage you to hang in there - it’s really worth it. There are less than a handful of people who understand the full stack of AI, from chip design to model architecture, as well as Reiner. It was a real delight to learn from him. Recommend watching this one on YouTube so you can see the chalkboard. 0:00:00 – How batch size affects token cost and speed 0:31:59 – How MoE models are laid out across GPU racks 0:47:02 – How pipeline parallelism spreads model layers across racks 1:03:27 – Why Ilya said, “As we now know, pipelining is not wise.” 1:18:49 – Because of RL, models may be 100x over-trained beyond Chinchilla-optimal 1:32:52 – Deducing long context memory costs from API pricing 2:03:52 – Convergent evolution between neural nets and cryptography
2:13:40

Did a very different format with Reiner Pope – a blackboard lecture where he walks through how frontier LLMs are trained and served. It's shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a bit technical, but I encourage you to hang in there - it’s really worth it. There are less than a handful of people who understand the full stack of AI, from chip design to model architecture, as well as Reiner. It was a real delight to learn from him. Recommend watching this one on YouTube so you can see the chalkboard. 0:00:00 – How batch size affects token cost and speed 0:31:59 – How MoE models are laid out across GPU racks 0:47:02 – How pipeline parallelism spreads model layers across racks 1:03:27 – Why Ilya said, “As we now know, pipelining is not wise.” 1:18:49 – Because of RL, models may be 100x over-trained beyond Chinchilla-optimal 1:32:52 – Deducing long context memory costs from API pricing 2:03:52 – Convergent evolution between neural nets and cryptography

Dwarkesh Patel

1,291,043 görüntüleme • 3 ay önce

Common Wealth: A unified theory of society in the age of AI Four works on personhood, economics, law, and the state. One continuous argument for how to build a better society from first principles.
1:14

Common Wealth: A unified theory of society in the age of AI Four works on personhood, economics, law, and the state. One continuous argument for how to build a better society from first principles.

Intelligent Internet

28,198 görüntüleme • 18 gün önce

Pope Francis Declares ‘Jesus is Satan’, Vows To Usher in ‘One World Religion’ Pope Francis is one of the most hated men in the world today. Those who hate him most are not atheists, or protestants, or Muslims, but Christians and his own followers. Notice the Pope is reading from a script? Who wrote the script…
11:05

Pope Francis Declares ‘Jesus is Satan’, Vows To Usher in ‘One World Religion’ Pope Francis is one of the most hated men in the world today. Those who hate him most are not atheists, or protestants, or Muslims, but Christians and his own followers. Notice the Pope is reading from a script? Who wrote the script…

The Punisher

1,385,646 görüntüleme • 2 yıl önce

CNN TO BERNIE: “You are one of the most widely admired Americans. I think you're only second to the Pope!”
0:52

CNN TO BERNIE: “You are one of the most widely admired Americans. I think you're only second to the Pope!”

Breaking911

959,869 görüntüleme • 1 yıl önce

The Hound was one character whom I enjoyed watching from the beginning to the end
0:57

The Hound was one character whom I enjoyed watching from the beginning to the end

Unique Movie Moments 🐬

74,940 görüntüleme • 9 ay önce

One year has passed since the white smoke of 8 May 2025 announced to the world the election of Pope Francis’ successor. Our video retraces the most significant moments of the American Pontiff’s first year, from audiences to apostolic journeys to Türkiye, Lebanon, and Africa.
1:53

One year has passed since the white smoke of 8 May 2025 announced to the world the election of Pope Francis’ successor. Our video retraces the most significant moments of the American Pontiff’s first year, from audiences to apostolic journeys to Türkiye, Lebanon, and Africa.

Vatican News

35,831 görüntüleme • 3 ay önce

Nick Reiner, the troubled son of slain Hollywood filmmaker Rob Reiner, pleaded not guilty to murder charges stemming from the fatal stabbing of his parents in their home, one of the most shocking celebrity homicide cases in Los Angeles history
0:41

Nick Reiner, the troubled son of slain Hollywood filmmaker Rob Reiner, pleaded not guilty to murder charges stemming from the fatal stabbing of his parents in their home, one of the most shocking celebrity homicide cases in Los Angeles history

Reuters

73,189 görüntüleme • 6 ay önce

From one of the most hated song to one of the favorite track from #Karuppu 🔥🤌🏻
0:30

From one of the most hated song to one of the favorite track from #Karuppu 🔥🤌🏻

Kerala Box Office

29,333 görüntüleme • 3 ay önce

On a chip, even something as basic as selecting one input from a register file has to be built out of AND and OR gates wired together. Simple-seeming operations end up surprisingly expensive. Reiner Pope showed me how this stacks up. In older generations of chips, most of the die area was used for shuffling data around these circuits, not for the actual compute! This is one of the problems modern chip design tries to solve.
4:07

On a chip, even something as basic as selecting one input from a register file has to be built out of AND and OR gates wired together. Simple-seeming operations end up surprisingly expensive. Reiner Pope showed me how this stacks up. In older generations of chips, most of the die area was used for shuffling data around these circuits, not for the actual compute! This is one of the problems modern chip design tries to solve.

Dwarkesh Patel

27,876 görüntüleme • 3 ay önce

Nick Reiner faces charges of parricide — the killing of a parent by their child. It is one of the rarest crimes, and one of the most disturbing because it violates one of the most basic human bonds. Parents are supposed to protect their children. Children are not supposed to fear — or kill — their parents. This video examines the Nick Reiner murder trial, the meaning of parricide, and the warning signs and psychological factors experts say often precede these rare cases.
17:21

Nick Reiner faces charges of parricide — the killing of a parent by their child. It is one of the rarest crimes, and one of the most disturbing because it violates one of the most basic human bonds. Parents are supposed to protect their children. Children are not supposed to fear — or kill — their parents. This video examines the Nick Reiner murder trial, the meaning of parricide, and the warning signs and psychological factors experts say often precede these rare cases.

Dr. Phil

32,680 görüntüleme • 7 ay önce

One of the most incredible, eloquent, intelligent, powerful and inspiring videos I’ve ever enjoyed from near the beginning of the war. This is the incredible Noa Tishby
9:05

One of the most incredible, eloquent, intelligent, powerful and inspiring videos I’ve ever enjoyed from near the beginning of the war. This is the incredible Noa Tishby

Chaya’s Clan

120,532 görüntüleme • 2 yıl önce

Let’s break down one of the best hitters in the country and steal what works. Brenden Lawson had one of the most impressive bats I saw all year. #2 in OPS in the whole SEC. What stood out the most? And what can your hitters actually take from it? Drop it below. 📝👇🏼
4:47

Let’s break down one of the best hitters in the country and steal what works. Brenden Lawson had one of the most impressive bats I saw all year. #2 in OPS in the whole SEC. What stood out the most? And what can your hitters actually take from it? Drop it below. 📝👇🏼

Trey Hannam

59,215 görüntüleme • 3 ay önce