Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

.Reiner Pope explains the difference between a CPU and a GPU. A huge chunk of a CPU die is actually just predicting what code will run next. GPUs strip this out, which is part of why they can fit so much more compute on the chip.

Dwarkesh Patel

252,167 subscribers

62,765 views • 3 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Breaking down the difference between CPU and GPU. This is such a joy to watch.

Breaking down the difference between CPU and GPU. This is such a joy to watch.

Barsee 🐶

55,020 views • 1 year ago

Incoming: The Big Ole Chip Flip CPUs > GPUs COATUE CIO of Public Investments Jaimin Rangwalla says AI agents could fundamentally reshape compute demand: “Historically CPUs have done something called serial processing, which is like you just give it a set of instructions and it does one by one by one by one. It was very much a 1 CPU : 8 GPUs. It had gone up to 1 CPU : 16 GPUs, which was all of the math & all of the computation happening at the GPUs. But now the ratio is actually moving from 1 CPU : 4 GPUs."

Incoming: The Big Ole Chip Flip CPUs > GPUs COATUE CIO of Public Investments Jaimin Rangwalla says AI agents could fundamentally reshape compute demand: “Historically CPUs have done something called serial processing, which is like you just give it a set of instructions and it does one by one by one by one. It was very much a 1 CPU : 8 GPUs. It had gone up to 1 CPU : 16 GPUs, which was all of the math & all of the computation happening at the GPUs. But now the ratio is actually moving from 1 CPU : 4 GPUs."

Molly O’Shea

15,390 views • 3 months ago

"The GPU has a lot of tiny, tiny TPUs tiled across the whole chip." Reiner Pope on the architectural difference between the two dominant AI chip designs.

"The GPU has a lot of tiny, tiny TPUs tiled across the whole chip." Reiner Pope on the architectural difference between the two dominant AI chip designs.

Dwarkesh Patel

79,018 views • 3 months ago

Billions of times a second, all the circuitry on an AI chip pauses, just for a moment. Why? Reiner Pope explained to me what's going on, and what the clock cycle of a chip is actually telling us:

Billions of times a second, all the circuitry on an AI chip pauses, just for a moment. Why? Reiner Pope explained to me what's going on, and what the clock cycle of a chip is actually telling us:

Dwarkesh Patel

28,938 views • 3 months ago

What is a DPU, NPU, GPU vs CPU? And why are they in Cisco switches? #Sponsored #Cisco #dpu #CPU #npu #GPU #switch #CiscoLiveEMEA Cisco Live EMEA Cisco

What is a DPU, NPU, GPU vs CPU? And why are they in Cisco switches? #Sponsored #Cisco #dpu #CPU #npu #GPU #switch #CiscoLiveEMEA Cisco Live EMEA Cisco

David Bombal

14,665 views • 1 year ago

Every year I use ZERO "Run plays" for the CPU in #NBA2K26 and a lot of people ask why. People think it causes the CPU to never run plays, but as you can see if you can balance the individual player tendencies correctly the result is a smooth authentic CPU offensive experience.

Every year I use ZERO "Run plays" for the CPU in #NBA2K26 and a lot of people ask why. People think it causes the CPU to never run plays, but as you can see if you can balance the individual player tendencies correctly the result is a smooth authentic CPU offensive experience.

Shady 🧊

184,692 views • 5 months ago

Microsoft sold every spare CPU it had to Anthropic and OpenAI. Amazon tripled its CPU buys year over year and still can't keep up. Two of AWS's biggest customers asked Andy Jassy if they could buy the entire 2026 production run of Graviton chips. He said no. The ratio inside an AI datacenter used to be 100 megawatts of GPUs to 1 megawatt of CPUs. CPUs handled storage, checkpointing, pre-processing. Light work. GPUs did the actual training and inference. Then OpenAI shipped o1-preview in September 2024. RL post-training went from "check the model output with a regex" to "run classifiers" to "compile the code and run the unit tests" to "spin up a sandbox, call three databases, run a physics simulation, verify the answer." Every rollout now needs a CPU-backed environment to verify against. Codex 5.4 runs agentically for 6-7 hours at a time. Each database call, each cron job, each scraped URL is CPU work. Coding agent revenue went from a couple billion to north of $10B in six months. That compute is sitting on CPUs. The CPU to GPU ratio is now approaching 1:1. The entire global cloud was built for 1:8. That's why GitHub has been unstable for weeks. Nvidia and Arm both announced they're entering the server CPU market in March. TSMC will only meet 80% of server CPU wafer demand this year. High-end server CPU prices are already up 50%. When the GPU king and the IP licensor both pivot to CPUs in the same month, the boring chip isn't boring anymore.

Microsoft sold every spare CPU it had to Anthropic and OpenAI. Amazon tripled its CPU buys year over year and still can't keep up. Two of AWS's biggest customers asked Andy Jassy if they could buy the entire 2026 production run of Graviton chips. He said no. The ratio inside an AI datacenter used to be 100 megawatts of GPUs to 1 megawatt of CPUs. CPUs handled storage, checkpointing, pre-processing. Light work. GPUs did the actual training and inference. Then OpenAI shipped o1-preview in September 2024. RL post-training went from "check the model output with a regex" to "run classifiers" to "compile the code and run the unit tests" to "spin up a sandbox, call three databases, run a physics simulation, verify the answer." Every rollout now needs a CPU-backed environment to verify against. Codex 5.4 runs agentically for 6-7 hours at a time. Each database call, each cron job, each scraped URL is CPU work. Coding agent revenue went from a couple billion to north of $10B in six months. That compute is sitting on CPUs. The CPU to GPU ratio is now approaching 1:1. The entire global cloud was built for 1:8. That's why GitHub has been unstable for weeks. Nvidia and Arm both announced they're entering the server CPU market in March. TSMC will only meet 80% of server CPU wafer demand this year. High-end server CPU prices are already up 50%. When the GPU king and the IP licensor both pivot to CPUs in the same month, the boring chip isn't boring anymore.

Aakash Gupta

290,945 views • 4 months ago

$AMD will become a multi-trillion dollar company. The AI boom is no longer just about GPUs. In early AI data centers, the CPU-to-GPU ratio was around 1:8. Now it’s moving closer to 1:1 because agentic AI requires dramatically more CPU compute. Lisa Su: “Based on the demand signals we are seeing today and the structural increase in CPU compute requirements driven by agentic AI, we now expect the server CPU TAM to grow at greater than 35% annually, reaching over $120 billion by 2030.” Add to that the 6GW AI infrastructure deals with $META and OpenAI, plus deep collaboration on multiple future chip generations, and it becomes clear why $AMD is positioned to compound for years. I’m not chasing $AMD here after a +110% move in just over a month. But the long-term story remains intact.

$AMD will become a multi-trillion dollar company. The AI boom is no longer just about GPUs. In early AI data centers, the CPU-to-GPU ratio was around 1:8. Now it’s moving closer to 1:1 because agentic AI requires dramatically more CPU compute. Lisa Su: “Based on the demand signals we are seeing today and the structural increase in CPU compute requirements driven by agentic AI, we now expect the server CPU TAM to grow at greater than 35% annually, reaching over $120 billion by 2030.” Add to that the 6GW AI infrastructure deals with $META and OpenAI, plus deep collaboration on multiple future chip generations, and it becomes clear why $AMD is positioned to compound for years. I’m not chasing $AMD here after a +110% move in just over a month. But the long-term story remains intact.

Daniel

39,622 views • 3 months ago

Dylan Patel, founder of SemiAnalysis, on why GitHub keeps breaking: "The entire cloud market ran out of CPUs. Microsoft sold all their spare ones to Anthropic and OpenAI. They have none left." That GitHub instability you keep hitting isn't a bug. It's the AI labs eating the world's compute. One customer ran a million CPU jobs in six hours. Amazon tripled CPU server installs year on year and still ran dry. Everyone watches the GPU race. The thing that actually broke was the boring chip nobody was looking at.

Dylan Patel, founder of SemiAnalysis, on why GitHub keeps breaking: "The entire cloud market ran out of CPUs. Microsoft sold all their spare ones to Anthropic and OpenAI. They have none left." That GitHub instability you keep hitting isn't a bug. It's the AI labs eating the world's compute. One customer ran a million CPU jobs in six hours. Amazon tripled CPU server installs year on year and still ran dry. Everyone watches the GPU race. The thing that actually broke was the boring chip nobody was looking at.

zostaff

82,561 views • 2 months ago

Stress testing Metropolis 1998! - Population: 53,500 - CPU FPS: 80 - GPU FPS: 10-80 - RAM: 3,100 MB So: At the moment, the game is GPU bound. I know why and there's a lot of room for improvement. It will involve developing or customizing an existing GPU framework. There's also a lot of room to improve CPU/RAM, but right now it's in an okay place. #gamedev

Stress testing Metropolis 1998! - Population: 53,500 - CPU FPS: 80 - GPU FPS: 10-80 - RAM: 3,100 MB So: At the moment, the game is GPU bound. I know why and there's a lot of room for improvement. It will involve developing or customizing an existing GPU framework. There's also a lot of room to improve CPU/RAM, but right now it's in an okay place. #gamedev

Yesbox - Metropolis 1998

84,873 views • 1 year ago

Most C Programmers don't have a clue as to how the CPU works. Here is a lecture from my course on the C Language that introduces how to visualise the CPU as a C (or general software) Programmer.... More Details are at: Being able to model the behaviour of CPU provides a solid base to reason about your code. The summary is - - The CPU is a state machine which maintains it's states in registers. - It has busses to memory for requesting data/instruction read/writes. - Interrupts to attend to events. I discuss this and more in my course on the C Language.

Most C Programmers don't have a clue as to how the CPU works. Here is a lecture from my course on the C Language that introduces how to visualise the CPU as a C (or general software) Programmer.... More Details are at: Being able to model the behaviour of CPU provides a solid base to reason about your code. The summary is - - The CPU is a state machine which maintains it's states in registers. - It has busses to memory for requesting data/instruction read/writes. - Interrupts to attend to events. I discuss this and more in my course on the C Language.

Piyush Itankar

28,019 views • 1 year ago

ELON: TESLA'S AI5 CHIP IS THE SILICON VERSION OF “LESS IS MORE” "The AI5 chip will be 40 times better than the AI4 chip – not 40%, 40 times. We have a detailed understanding of the entire software and hardware stack. We are designing the hardware to address all of the pain points in software. We know what the chip needs to do, and we know, just as importantly, what the chip doesn’t need to do. A long list of deletions that are very important. As a result of these deletions, we can actually fit AI5 in a half reticle with a good margin for the traces from the memory to the Tesla chip accelerators, the ARM CPU cores, and the PCI blocks. This is a beautiful chip." Source: Elon Musk, Tesla Podcast, October 22, 2025

ELON: TESLA'S AI5 CHIP IS THE SILICON VERSION OF “LESS IS MORE” "The AI5 chip will be 40 times better than the AI4 chip – not 40%, 40 times. We have a detailed understanding of the entire software and hardware stack. We are designing the hardware to address all of the pain points in software. We know what the chip needs to do, and we know, just as importantly, what the chip doesn’t need to do. A long list of deletions that are very important. As a result of these deletions, we can actually fit AI5 in a half reticle with a good margin for the traces from the memory to the Tesla chip accelerators, the ARM CPU cores, and the PCI blocks. This is a beautiful chip." Source: Elon Musk, Tesla Podcast, October 22, 2025

Mario Nawfal

363,507 views • 10 months ago

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

Akshay 🚀

336,634 views • 9 days ago

NVIDIA CEO Jensen Huang announces the GB10 superchip GPU+CPU package for use with workstations packing 1 petaFLOPS of compute and a 4TB SSD

NVIDIA CEO Jensen Huang announces the GB10 superchip GPU+CPU package for use with workstations packing 1 petaFLOPS of compute and a 4TB SSD

Tsarathustra

131,621 views • 1 year ago

Ben Thompson says Nvidia is already cutting prices. The cut just doesn't show up in their margins: "Everyone's coming for them and they're still charging however much money for a chip. But they're not maintaining their margins, because who's buying? This circular financing people talk about, Nvidia's providing a 25% backstop. If you actually ascribe a value to Nvidia taking equity in the Neoclouds, they guarantee they're going to buy all their compute through 2030. And why do they do that? So the entity in question can get a lower cost of capital, so they can buy more GPUs. They get a lower cost of capital because Nvidia assumed the risk. Taking risk has a price. There's also a world where a ton of compute comes to market, the hyperscalers don't need this compute, and Nvidia's paying for compute that no one wants. That is a diminution of Nvidia's profitability if you look at their business holistically. What that is, is a price cut. The price just didn't show up in the margins."

Ben Thompson says Nvidia is already cutting prices. The cut just doesn't show up in their margins: "Everyone's coming for them and they're still charging however much money for a chip. But they're not maintaining their margins, because who's buying? This circular financing people talk about, Nvidia's providing a 25% backstop. If you actually ascribe a value to Nvidia taking equity in the Neoclouds, they guarantee they're going to buy all their compute through 2030. And why do they do that? So the entity in question can get a lower cost of capital, so they can buy more GPUs. They get a lower cost of capital because Nvidia assumed the risk. Taking risk has a price. There's also a world where a ton of compute comes to market, the hyperscalers don't need this compute, and Nvidia's paying for compute that no one wants. That is a diminution of Nvidia's profitability if you look at their business holistically. What that is, is a price cut. The price just didn't show up in the margins."

Patrick OShaughnessy

157,693 views • 13 days ago

.Beff (e/acc) explains how thermodynamic computing fits into the current computing stack: "We don't do GPUs. They're called TSUs, thermodynamic sampling units. We think people are gonna put them next to GPUs or whatever their favorite accelerator is." "You don't need to run the whole workload on the TSU, but it could run certain parts of the workload, just like the GPU is complementary to the CPU initially." "The TSU is gonna help give you more performance per watt in your data center when combined with existing build-outs." "Let's say you're serving a large diffusion model. What we showed is that by using a TSU, you can use something called an energy-based model instead of a simple probabilistic kernel." "Overall you can have basically 100 times the performance per watt of just running on pure GPUs because our chips are much lower wattage."

.Beff (e/acc) explains how thermodynamic computing fits into the current computing stack: "We don't do GPUs. They're called TSUs, thermodynamic sampling units. We think people are gonna put them next to GPUs or whatever their favorite accelerator is." "You don't need to run the whole workload on the TSU, but it could run certain parts of the workload, just like the GPU is complementary to the CPU initially." "The TSU is gonna help give you more performance per watt in your data center when combined with existing build-outs." "Let's say you're serving a large diffusion model. What we showed is that by using a TSU, you can use something called an energy-based model instead of a simple probabilistic kernel." "Overall you can have basically 100 times the performance per watt of just running on pure GPUs because our chips are much lower wattage."

MTS

42,570 views • 3 months ago

Dylan Patel: “CPUs are sold out.” $INTC earnings and conference call confirmed the looming CPU shortage. There are three main drivers: - Agentic systems, as CPUs do the planning and orchestration. - Increasing complexity of and demand for reinforcement learning environments that run on CPUs. - Increasing deployment of AI instilled apps. Models run on GPUs but apps themselves run on CPUs. Demand for all these will grow secularly as AI systems will only get more agentic, RL environments will only get more complex, and deployment will only accelerate. As a result, GPU-to-CPU ratio in AI workloads will move from 1:8 to 1:2 or maybe even 1:1 in the future. CPU supercycle is about to start. $AMD $INTC $ARM

Dylan Patel: “CPUs are sold out.” $INTC earnings and conference call confirmed the looming CPU shortage. There are three main drivers: - Agentic systems, as CPUs do the planning and orchestration. - Increasing complexity of and demand for reinforcement learning environments that run on CPUs. - Increasing deployment of AI instilled apps. Models run on GPUs but apps themselves run on CPUs. Demand for all these will grow secularly as AI systems will only get more agentic, RL environments will only get more complex, and deployment will only accelerate. As a result, GPU-to-CPU ratio in AI workloads will move from 1:8 to 1:2 or maybe even 1:1 in the future. CPU supercycle is about to start. $AMD $INTC $ARM

Oguz Erkan

93,419 views • 4 months ago

New blackboard lecture w Reiner Pope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost of data movement 0:25:59 – How systolic arrays work 0:39:00 – Clock cycles and pipeline registers 0:51:40 – FPGAs vs ASICs 1:03:14 – Cache vs scratchpad 1:07:16 – Why CPU cores are much bigger than GPU cores 1:11:49 – Brains vs chips 1:15:22 – A GPU is just a bunch of tiny TPUs Look up Dwarkesh Podcast on YouTube/Spotify/etc to watch. Enjoy!

New blackboard lecture w Reiner Pope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost of data movement 0:25:59 – How systolic arrays work 0:39:00 – Clock cycles and pipeline registers 0:51:40 – FPGAs vs ASICs 1:03:14 – Cache vs scratchpad 1:07:16 – Why CPU cores are much bigger than GPU cores 1:11:49 – Brains vs chips 1:15:22 – A GPU is just a bunch of tiny TPUs Look up Dwarkesh Podcast on YouTube/Spotify/etc to watch. Enjoy!

Dwarkesh Patel

937,539 views • 3 months ago

Fading out audio is one of the most CPU-intensive tasks you can possibly do! When numbers get really small, the number of CPU operations can explode 100-fold. In this LaurieWired video, we'll explore the IEEE 754 standard, the fight between Intel and DEC, and more!

Fading out audio is one of the most CPU-intensive tasks you can possibly do! When numbers get really small, the number of CPU operations can explode 100-fold. In this LaurieWired video, we'll explore the IEEE 754 standard, the fight between Intel and DEC, and more!

LaurieWired

403,177 views • 1 year ago