Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Why can't we have arbitrarily large GPU scale-ups? Reiner Pope explains it's partially a matter of wire density:

dwarkeshpodcast's profile picture

Dwarkesh Podcast

11,820 subscribers

19,447 Aufrufe • vor 5 Tagen •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

1 Kommentare

Profilbild von rohan anil
rohan anilvor 5 Tagen

@reinerpope Weakly connected compute island scale up does work (see the hitchhikers blog), taking it to extremes we could touch 1e28 flops, which is pretty neat. Of course coordination of such an enterprise is more complex than training algorithms.

Ähnliche Videos

.Reiner Pope explains the difference between a CPU and a GPU. A huge chunk of a CPU die is actually just predicting what code will run next. GPUs strip this out, which is part of why they can fit so much more compute on the chip.
2:49

.Reiner Pope explains the difference between a CPU and a GPU. A huge chunk of a CPU die is actually just predicting what code will run next. GPUs strip this out, which is part of why they can fit so much more compute on the chip.

Dwarkesh Patel

62,765 Aufrufe • vor 5 Monaten

"The GPU has a lot of tiny, tiny TPUs tiled across the whole chip." Reiner Pope on the architectural difference between the two dominant AI chip designs.
3:38

"The GPU has a lot of tiny, tiny TPUs tiled across the whole chip." Reiner Pope on the architectural difference between the two dominant AI chip designs.

Dwarkesh Patel

79,932 Aufrufe • vor 5 Monaten

How did Tensor Cores massively increase the throughput of Nvidia chips? Reiner Pope explains the fundamental idea, systolic arrays:
5:27

How did Tensor Cores massively increase the throughput of Nvidia chips? Reiner Pope explains the fundamental idea, systolic arrays:

Dwarkesh Patel

32,174 Aufrufe • vor 5 Monaten

Fuentes explains why The Daily Wire layoffs is a perfect example of how Jewish media works "If they don't have an audience, IT DOESN'T MATTER. The purpose is only to flood the zone with their messaging."
4:11

Fuentes explains why The Daily Wire layoffs is a perfect example of how Jewish media works "If they don't have an audience, IT DOESN'T MATTER. The purpose is only to flood the zone with their messaging."

Fuentes Updates

246,175 Aufrufe • vor 6 Monaten

ICYMI. "This is why we can't have nice things. Corruption." Rachel Maddow explains. WATCH: #DemsUnited
5:32

ICYMI. "This is why we can't have nice things. Corruption." Rachel Maddow explains. WATCH: #DemsUnited

BMcArthur

24,481 Aufrufe • vor 4 Monaten

Trust the process 🤝 Marc Márquez explains why a few push-ups have become part of his grid routine! 💪 #JapaneseGP🇯🇵
0:36

Trust the process 🤝 Marc Márquez explains why a few push-ups have become part of his grid routine! 💪 #JapaneseGP🇯🇵

MotoGP™🏁

66,885 Aufrufe • vor 1 Monat

🇺🇸🇺🇦🇷🇺 "For Trump, striking off Belgorod is not flashy. He thinks on a large scale. It's a different matter to have a blackout in Moscow. That's why they are now talking about 'Tomahawks,' they are preparing us for this strike," — political scientist Mikhail Sheitelman
1:25

🇺🇸🇺🇦🇷🇺 "For Trump, striking off Belgorod is not flashy. He thinks on a large scale. It's a different matter to have a blackout in Moscow. That's why they are now talking about 'Tomahawks,' they are preparing us for this strike," — political scientist Mikhail Sheitelman

Lord Bebo

55,002 Aufrufe • vor 1 Jahr

Using large-scale GPU parallelized reinforcement learning in Isaac Sim, we are able to obtain a robust whole-body controller for Taku optimized for precise loco-manipulation.
0:20

Using large-scale GPU parallelized reinforcement learning in Isaac Sim, we are able to obtain a robust whole-body controller for Taku optimized for precise loco-manipulation.

Dyna Robotics

85,952 Aufrufe • vor 1 Monat

We finally got a real look at actual GTA 6 gameplay, and it's just as impressive as we'd hoped. The scale and density of Vice City look incredible. More details:
0:25

We finally got a real look at actual GTA 6 gameplay, and it's just as impressive as we'd hoped. The scale and density of Vice City look incredible. More details:

80 LEVEL

12,395 Aufrufe • vor 2 Monaten

.Jack Mallers explains why he questioned Saylor on MNAV and dilution at Bitcoin Prague. "If there's no definitions to any of these things, we can't have a conversation."
4:31

.Jack Mallers explains why he questioned Saylor on MNAV and dilution at Bitcoin Prague. "If there's no definitions to any of these things, we can't have a conversation."

TFTC

18,906 Aufrufe • vor 4 Monaten

Nick explains why we don’t have a Wii yet
2:15

Nick explains why we don’t have a Wii yet

OregonGroyper

22,926 Aufrufe • vor 1 Monat

.Jonathan Reiner 's take on the future of Pope Leo since we now know he he has lung cancer. 10.13.26
4:56

.Jonathan Reiner 's take on the future of Pope Leo since we now know he he has lung cancer. 10.13.26

Jeff Storobinsky

69,615 Aufrufe • vor 27 Tagen

"We have normalized a situation in which it's normal to kill doctors, it's normal to kill nurses, it's normal to bomb hospitals from the air, with tanks... anything you like." Dr. Feroze Sidhwa on the "large-scale killing" of children & destruction of healthcare in Gaza.
1:44

"We have normalized a situation in which it's normal to kill doctors, it's normal to kill nurses, it's normal to bomb hospitals from the air, with tanks... anything you like." Dr. Feroze Sidhwa on the "large-scale killing" of children & destruction of healthcare in Gaza.

AYMAN

276,892 Aufrufe • vor 2 Jahren

.Lindy explains why sign ups are down
0:47

.Lindy explains why sign ups are down

Jason

92,252 Aufrufe • vor 2 Monaten

WATCH - Canada's military chief: "We have to get ready for large-scale conflicts"
0:41

WATCH - Canada's military chief: "We have to get ready for large-scale conflicts"

Insider Paper

53,114 Aufrufe • vor 7 Monaten

RIP Rob Reiner and Michelle Reiner. It's unbelievable that we have allowed this sick, vile fuck [Trump] to live in the People's House. 😳👇
1:01

RIP Rob Reiner and Michelle Reiner. It's unbelievable that we have allowed this sick, vile fuck [Trump] to live in the People's House. 😳👇

Bill Madden

25,825 Aufrufe • vor 10 Monaten

Experimenting with a new #GenerativeAI #Photoshop plugin. It's currently limited to layer operations, but there's no reason why it can't scale to all of Ps.
1:40

Experimenting with a new #GenerativeAI #Photoshop plugin. It's currently limited to layer operations, but there's no reason why it can't scale to all of Ps.

Christian Cantrell

11,810 Aufrufe • vor 3 Jahren

Trump: "It's actually a matter in a serious way of national survival. You can't have these elections going on like this anymore."
0:22

Trump: "It's actually a matter in a serious way of national survival. You can't have these elections going on like this anymore."

Aaron Rupar

152,839 Aufrufe • vor 8 Monaten