Loading video...

Video Failed to Load

Go Home

Gavin Baker says the piece missing from the compute debate is SRAM accelerators, and disaggregating inference across three chips lifts AI's ROI. "I do think something that is missing from all of this conversation about compute is what is going to happen when you put these SRAM-based accelerators that...

firesidealpha's profile picture

Fireside Alpha

12,222 subscribers

15,501 views • 18 days ago •via X (Twitter)

Science & TechnologyNews & Politics
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Etched is deploying two new technologies in chip design: low-voltage inference and cluster-scale memory. CEO Gavin Uberti says they'll make their chips much more power-efficient and way, way faster than today's leading GPUs. He breaks it down: "We looked at a lot of early research directions, and we realized the key things that models need are way more compute and way faster memory." "If you think about inference, there are two key parts: prefill and decode. For prefill, it's a compute-bound problem. You need to have more FLOPS, more operations per second on each of your chips." "On our GPU, the bottleneck's actually thermals. You can't really run a GPU at more than around 50% of what it could theoretically do, or it'll melt." "So we're using a new technology today called low-voltage inference to try to solve this problem. You bring the voltage of the chip down dramatically, which allows us to have way, way better efficiency in terms of how much power is drawn per unit of math, and thus fit way way more flops onto the chip..." "For decode, it's all about bandwidth. Not just bandwidth on a chip, but bandwidth across your cluster. That's why we have this technology we call cluster-scale memory. It reduces the amount of time it takes to communicate from one chip to another dramatically." "As a result we can go use all of our HBM, HBM bandwidth, SRAM, SRAM bandwidth, and our scale-up domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second, while still keeping your costs low."
3:19

Etched is deploying two new technologies in chip design: low-voltage inference and cluster-scale memory. CEO Gavin Uberti says they'll make their chips much more power-efficient and way, way faster than today's leading GPUs. He breaks it down: "We looked at a lot of early research directions, and we realized the key things that models need are way more compute and way faster memory." "If you think about inference, there are two key parts: prefill and decode. For prefill, it's a compute-bound problem. You need to have more FLOPS, more operations per second on each of your chips." "On our GPU, the bottleneck's actually thermals. You can't really run a GPU at more than around 50% of what it could theoretically do, or it'll melt." "So we're using a new technology today called low-voltage inference to try to solve this problem. You bring the voltage of the chip down dramatically, which allows us to have way, way better efficiency in terms of how much power is drawn per unit of math, and thus fit way way more flops onto the chip..." "For decode, it's all about bandwidth. Not just bandwidth on a chip, but bandwidth across your cluster. That's why we have this technology we call cluster-scale memory. It reduces the amount of time it takes to communicate from one chip to another dramatically." "As a result we can go use all of our HBM, HBM bandwidth, SRAM, SRAM bandwidth, and our scale-up domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second, while still keeping your costs low."

TBPN

20,404 views • 1 month ago

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."
1:12

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Fireside Alpha

282,905 views • 27 days ago

Chamath: Two terms you need to pay attention to in AI are Prefill and Decode “There's two terms that I think you're going to hear a ton about over these next few years.” “The first term is prefill, and the next is decode.” “What prefill and decode are, are two very distinct ways of how models think, and how a model goes through the process of answering a question that you ask it.” “And so when you send a prompt to AI, what happens is that the model processes it. This is called the reading phase or prefill.” “It reads your entire prompt all at once. And then it does a bunch of math, calculates all these relationships between all the words, and it stores them in temporary memory.” “The problem is that this is really compute bound. So it requires massive brute force. And Nvidia GPUs crush here.” “And their architecture is designed for massive parallel processing, which makes them really amazing at digesting these long prompts.” “So the problem just gets bigger and bigger, Nvidia just completely dominates.” “But the next phase though, this critical phase, the decode phase, is the writing phase, right?” “So the model starts to generate a response, you ask it a question and its response, one token at a time.” “And then to pick the next token to pick the next word, it has to look back at everything it has said already so that it doesn't hallucinate.” “The problem is that this is incredibly memory bandwidth constrained.” “And in our architecture, a long time ago, we made these design decisions from day one.” “And so what we did was we took a very different architectural approach, we took a very conservative process technology. We weren't pushing the boundaries of physics.” “And we used a lot of what's called SRAM. So memory on the chip so that we could do this decode thing as well or better than everybody else.” “And so now when you put these two things together, I just think it's going to create a huge acceleration in the ability for this entire infrastructure layer to get much cheaper and much more valuable, which I suspect then it'll have a lot more developer pull, you'll get a lot more applications being built, billions and billions of more people using it.”
1:52

Chamath: Two terms you need to pay attention to in AI are Prefill and Decode “There's two terms that I think you're going to hear a ton about over these next few years.” “The first term is prefill, and the next is decode.” “What prefill and decode are, are two very distinct ways of how models think, and how a model goes through the process of answering a question that you ask it.” “And so when you send a prompt to AI, what happens is that the model processes it. This is called the reading phase or prefill.” “It reads your entire prompt all at once. And then it does a bunch of math, calculates all these relationships between all the words, and it stores them in temporary memory.” “The problem is that this is really compute bound. So it requires massive brute force. And Nvidia GPUs crush here.” “And their architecture is designed for massive parallel processing, which makes them really amazing at digesting these long prompts.” “So the problem just gets bigger and bigger, Nvidia just completely dominates.” “But the next phase though, this critical phase, the decode phase, is the writing phase, right?” “So the model starts to generate a response, you ask it a question and its response, one token at a time.” “And then to pick the next token to pick the next word, it has to look back at everything it has said already so that it doesn't hallucinate.” “The problem is that this is incredibly memory bandwidth constrained.” “And in our architecture, a long time ago, we made these design decisions from day one.” “And so what we did was we took a very different architectural approach, we took a very conservative process technology. We weren't pushing the boundaries of physics.” “And we used a lot of what's called SRAM. So memory on the chip so that we could do this decode thing as well or better than everybody else.” “And so now when you put these two things together, I just think it's going to create a huge acceleration in the ability for this entire infrastructure layer to get much cheaper and much more valuable, which I suspect then it'll have a lot more developer pull, you'll get a lot more applications being built, billions and billions of more people using it.”

The All-In Podcast

567,546 views • 7 months ago

The CEO of the world's largest asset manager just said something that should reframe how every investor thinks about the AI trade. Larry Fink, managing $11.5 trillion at BlackRock, stood at the Milken Institute Global Conference and said four words that matter, "We just don't have enough compute." "The United States is short power. We're short compute. We're short chips. And there's going to be shortages in all three and memory, four things. I actually believe a new asset class will be buying futures of compute." Think about what that means. Fink is predicting that compute becomes a tradable commodity like oil, like grain, like natural gas where investors buy forward contracts on future capacity because the shortage is so structural and so predictable that a derivatives market will emerge to price it. That is not a minor observation from a finance executive but rather the chairman of the most powerful capital allocator on the planet telling you that compute scarcity is a multi-year, investable megatrend. The data backs him up completely. Data centers will consume 70% of all memory chips produced globally in 2026. Advanced HBM production from Samsung, SK Hynix, and Micron is sold out through 2026 and into 2027 and a single AI server consumes 10-20x more memory than a conventional workload server. DRAM supply growth is running at just 16% annually while AI infrastructure demand is growing at 80%+. The chip crunch, the power crunch, and the compute crunch are not temporary dislocations, they are structural, and they will get worse before they get better. Fink also said something the bears keep getting wrong: "There is not an AI bubble. There is the opposite. We have supply shortages. Demand is growing much faster than anyone has ever anticipated." This is why the Milk Road Pro portfolio is built the way it is, long the companies producing and supplying the constrained resources: chips, memory, compute infrastructure, and power. Check out Milk Road Pro, link below to access our full thesis and plays.
0:23

The CEO of the world's largest asset manager just said something that should reframe how every investor thinks about the AI trade. Larry Fink, managing $11.5 trillion at BlackRock, stood at the Milken Institute Global Conference and said four words that matter, "We just don't have enough compute." "The United States is short power. We're short compute. We're short chips. And there's going to be shortages in all three and memory, four things. I actually believe a new asset class will be buying futures of compute." Think about what that means. Fink is predicting that compute becomes a tradable commodity like oil, like grain, like natural gas where investors buy forward contracts on future capacity because the shortage is so structural and so predictable that a derivatives market will emerge to price it. That is not a minor observation from a finance executive but rather the chairman of the most powerful capital allocator on the planet telling you that compute scarcity is a multi-year, investable megatrend. The data backs him up completely. Data centers will consume 70% of all memory chips produced globally in 2026. Advanced HBM production from Samsung, SK Hynix, and Micron is sold out through 2026 and into 2027 and a single AI server consumes 10-20x more memory than a conventional workload server. DRAM supply growth is running at just 16% annually while AI infrastructure demand is growing at 80%+. The chip crunch, the power crunch, and the compute crunch are not temporary dislocations, they are structural, and they will get worse before they get better. Fink also said something the bears keep getting wrong: "There is not an AI bubble. There is the opposite. We have supply shortages. Demand is growing much faster than anyone has ever anticipated." This is why the Milk Road Pro portfolio is built the way it is, long the companies producing and supplying the constrained resources: chips, memory, compute infrastructure, and power. Check out Milk Road Pro, link below to access our full thesis and plays.

Milk Road AI

419,283 views • 3 months ago

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.
3:14

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

MTS

44,057 views • 3 months ago

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.
7:57

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Brian Roemmele

87,214 views • 2 years ago

If $MU says NAND pricing is still strong, enterprise SSD demand is picking up and AI inference is creating more storage demand then $SNDK should benefit because SanDisk is one of the more direct ways to own that part of the cycle. Hyperscalers are still figuring out the right mix of DRAM, HBM, TLC NAND, QLC NAND and other storage architectures which can keep supply tighter for longer as memory companies are going to be hesitant to add too much capacity before demand becomes clearer. I'm going to be listening closely for Micron’s QLC commentary because QLC NAND for read-heavy inference workloads is the big driver and if Micron calls that out then it would show that the storage side of the AI memory story is still earlier than DRAM.
1:03

If $MU says NAND pricing is still strong, enterprise SSD demand is picking up and AI inference is creating more storage demand then $SNDK should benefit because SanDisk is one of the more direct ways to own that part of the cycle. Hyperscalers are still figuring out the right mix of DRAM, HBM, TLC NAND, QLC NAND and other storage architectures which can keep supply tighter for longer as memory companies are going to be hesitant to add too much capacity before demand becomes clearer. I'm going to be listening closely for Micron’s QLC commentary because QLC NAND for read-heavy inference workloads is the big driver and if Micron calls that out then it would show that the storage side of the AI memory story is still earlier than DRAM.

Shay Boloor

108,953 views • 1 month ago

Peter Thiel on $NVDA (about a year ago): It is probably quite tricky. If you had to concretize it, one thing that is very strange is if you just follow the money, at this point 80 to 85% of the money in AI is being made by one company, it is NVIDIA. It is all on this very weird hardware layer, which Silicon Valley does not even know very much about anymore. We do not really do hardware, we do not do silicon chips in Silicon Valley anymore. I get pitched on these companies once every three or four years, and it is always, I have no clue how to do this, it sounds like a pretty good idea, but man, I have no clue, and we never invest. There is this theory that the hardware piece makes the money initially, then gets more commodified over time, and it will shift to software. And the, I do not know, multi trillion dollar question is whether that is going to be true again this time, or whether NVIDIA will have this incredible monopoly. I suspect NVIDIA will. I think it will maintain its position for a while. I think the game theory on it is something like this. All the big tech companies are going to start trying to design their own AI chips so they do not have to pay the 10x markup to NVIDIA. How hard is it for them to do it? How long will it take? If they all do it, then the chips become a commodity and nobody makes money in chips. So do you go into hardware? You should do it if nobody else is doing it. If everybody does it, you should not do it. I am not sure how that nets out, but probably people stay stuck for a while and NVIDIA goes from strength to strength for a while.
1:55

Peter Thiel on $NVDA (about a year ago): It is probably quite tricky. If you had to concretize it, one thing that is very strange is if you just follow the money, at this point 80 to 85% of the money in AI is being made by one company, it is NVIDIA. It is all on this very weird hardware layer, which Silicon Valley does not even know very much about anymore. We do not really do hardware, we do not do silicon chips in Silicon Valley anymore. I get pitched on these companies once every three or four years, and it is always, I have no clue how to do this, it sounds like a pretty good idea, but man, I have no clue, and we never invest. There is this theory that the hardware piece makes the money initially, then gets more commodified over time, and it will shift to software. And the, I do not know, multi trillion dollar question is whether that is going to be true again this time, or whether NVIDIA will have this incredible monopoly. I suspect NVIDIA will. I think it will maintain its position for a while. I think the game theory on it is something like this. All the big tech companies are going to start trying to design their own AI chips so they do not have to pay the 10x markup to NVIDIA. How hard is it for them to do it? How long will it take? If they all do it, then the chips become a commodity and nobody makes money in chips. So do you go into hardware? You should do it if nobody else is doing it. If everybody does it, you should not do it. I am not sure how that nets out, but probably people stay stuck for a while and NVIDIA goes from strength to strength for a while.

Wall St Engine

824,912 views • 9 months ago

💡" When I think about the revolution and I think about how I talk to people, it really is about the fact that you're moving money, with Sub-Zero fees, like the fees are just tiny, and so it allows just so much opportunity for growth to be built on top of the network, and that truly is what an open network is all about." - Denelle Dixon Full interview on PayPal 🤝 Stellar's announcement 👇
10:29

💡" When I think about the revolution and I think about how I talk to people, it really is about the fact that you're moving money, with Sub-Zero fees, like the fees are just tiny, and so it allows just so much opportunity for growth to be built on top of the network, and that truly is what an open network is all about." - Denelle Dixon Full interview on PayPal 🤝 Stellar's announcement 👇

CoinDesk

62,290 views • 1 year ago

“Sometimes the thought that we have as a hitter is different from the reality of how our body thinks to do something…There are times when we do high spin fastballs in here with players, where they’re swinging and missing. And then I say on the next one, I want you to try and swing and miss above the ball. And then what do you think happens on the next one? They hit it”🧠🤯 Listen to Eugene talk about why a lot of players talk about swinging down especially on high ride fastballs and how the thought doesn’t mean it’s exactly how their body is going to move🗣️ #the108way
1:25

“Sometimes the thought that we have as a hitter is different from the reality of how our body thinks to do something…There are times when we do high spin fastballs in here with players, where they’re swinging and missing. And then I say on the next one, I want you to try and swing and miss above the ball. And then what do you think happens on the next one? They hit it”🧠🤯 Listen to Eugene talk about why a lot of players talk about swinging down especially on high ride fastballs and how the thought doesn’t mean it’s exactly how their body is going to move🗣️ #the108way

108 Performance

17,811 views • 2 years ago

🚨 🚨 🇱🇧 🇺🇸 Secretary Rubio on Lebanon: - And so how do you defeat Hizballah? How do you replace Hizballah? You defeat and replace them with a government that is strong enough to be the sole force of arms in the country, basically with the security of Lebanon in the hands of the Lebanese Armed Forces, and a government that’s able to provide prosperity for its citizens. So we talked about the opportunities to do that. This is not just about miliary stuff. This is about how can we attract more U.S. and international investment into Lebanon. This is about things like how can we restart direct flights between the United States and Lebanon that are – right now, you can’t do that. So we’re going to start a process with them and see if that’s possible and when that can be done. -So we had a good meeting, and I think he’ll leave on Tuesday. He’ll have a very positive meeting with President Trump. And I think President Aoun will be able to return to Lebanon and say to the people of Lebanon that he is the one that’s getting positive results for the people
1:31

🚨 🚨 🇱🇧 🇺🇸 Secretary Rubio on Lebanon: - And so how do you defeat Hizballah? How do you replace Hizballah? You defeat and replace them with a government that is strong enough to be the sole force of arms in the country, basically with the security of Lebanon in the hands of the Lebanese Armed Forces, and a government that’s able to provide prosperity for its citizens. So we talked about the opportunities to do that. This is not just about miliary stuff. This is about how can we attract more U.S. and international investment into Lebanon. This is about things like how can we restart direct flights between the United States and Lebanon that are – right now, you can’t do that. So we’re going to start a process with them and see if that’s possible and when that can be done. -So we had a good meeting, and I think he’ll leave on Tuesday. He’ll have a very positive meeting with President Trump. And I think President Aoun will be able to return to Lebanon and say to the people of Lebanon that he is the one that’s getting positive results for the people

ToniMrad

27,695 views • 1 month ago

This is one of my favorite movie endings... It has a powerful lesson: "At some point, everything is going to go south on you...and you're gonna say 'this is it, this is how I end.' Now you can either accept that, or you can get to work. That's all it is, you just begin. You do the math. You solve one problem. Then you solve the next one. And the next. And if you solve enough problems, you get to come home." This is the beautiful thing about life: No matter where you are today—no matter how deep in the darkness—you are always just one good decision away from being in a better place tomorrow. You can always focus on that one decision—that one problem—in front of you. Don't worry about the hundreds or thousands of decisions that you still have to make to get to where you want to be—just focus on the next decision. Just begin. Solve one problem. Then solve the next one. And the next. "Life is really simple, but we insist on making it complicated." - Confucius
0:50

This is one of my favorite movie endings... It has a powerful lesson: "At some point, everything is going to go south on you...and you're gonna say 'this is it, this is how I end.' Now you can either accept that, or you can get to work. That's all it is, you just begin. You do the math. You solve one problem. Then you solve the next one. And the next. And if you solve enough problems, you get to come home." This is the beautiful thing about life: No matter where you are today—no matter how deep in the darkness—you are always just one good decision away from being in a better place tomorrow. You can always focus on that one decision—that one problem—in front of you. Don't worry about the hundreds or thousands of decisions that you still have to make to get to where you want to be—just focus on the next decision. Just begin. Solve one problem. Then solve the next one. And the next. "Life is really simple, but we insist on making it complicated." - Confucius

Sahil Bloom

213,822 views • 2 years ago

Leopold explains his approach to not blowing up (2024) "Obviously, not blowing up is task number one and two or whatever. I think this investment firm is going to just be betting on AGI and superintelligence before the decade is out, making the bets you would make if you took that seriously." "I think if that's wrong, the firm is not going to do that well." "The thing you have to be able to resist is, you know, one or a couple or a few individual calls. It's like AI stagnates for a year because of the data wall, or you got the call wrong on when revenue would go up." "You have to get timing right. I do think in general that the sequence of bets on the way to AGI is actually pretty critical, and something people underrate."
0:42

Leopold explains his approach to not blowing up (2024) "Obviously, not blowing up is task number one and two or whatever. I think this investment firm is going to just be betting on AGI and superintelligence before the decade is out, making the bets you would make if you took that seriously." "I think if that's wrong, the firm is not going to do that well." "The thing you have to be able to resist is, you know, one or a couple or a few individual calls. It's like AI stagnates for a year because of the data wall, or you got the call wrong on when revenue would go up." "You have to get timing right. I do think in general that the sequence of bets on the way to AGI is actually pretty critical, and something people underrate."

dnap

2,327,691 views • 23 days ago

Adaption Labs CEO Sara Hooker says data centers in space are "pretty bonkers", and explains what the current consensus gets wrong: "Most co-located hardware is pretty much for training. I think that's why you care [about Space], otherwise you can distribute. Inference compute, which is where everything's moving towards, you can spread that compute more easily. You can have multiple data centers [on Earth]." "So if you care about space, you probably only care about training compute. I think people underestimate the amount of failures that happen, and you don't want to get your training job interrupted." "The real issue is that GPUs still have failure rates. The 2% of GPUs that are just considered done every year, that's really your cost. It's how quickly you can replace those [in space] and what that looks like."
2:29

Adaption Labs CEO Sara Hooker says data centers in space are "pretty bonkers", and explains what the current consensus gets wrong: "Most co-located hardware is pretty much for training. I think that's why you care [about Space], otherwise you can distribute. Inference compute, which is where everything's moving towards, you can spread that compute more easily. You can have multiple data centers [on Earth]." "So if you care about space, you probably only care about training compute. I think people underestimate the amount of failures that happen, and you don't want to get your training job interrupted." "The real issue is that GPUs still have failure rates. The 2% of GPUs that are just considered done every year, that's really your cost. It's how quickly you can replace those [in space] and what that looks like."

TBPN

37,463 views • 6 months ago

If we get AGI before 2030 but can only produce ~200 GW a year of chips on advanced process nodes, can we ease the bottleneck by going back to older nodes? Some of the most singularity-pilled investors think so. Most of the FLOPS improvement between chip generations has come from numerics - not transistor density. But you could just design a new version of a 7nm chip like the A100 to have FP4. Holding numerics and die area constant, a B200 (2025 chip on 4nm) would only have ~3.5x more FLOPS than an A100 (2020 chip on 7nm). That seems like a manageable haircut - does it mean we can blow past the 200 GW ceiling? Dylan Patel explains why this is not an apples-to-apples comparison. If you run real-world performance analysis on DeepSeek V3 or Kimi K2.5, the performance gap between Hopper and Blackwell isn't 3x. It's ~20x. Why? Because inference is already memory-bandwidth-bound, and older chips have far fewer FLOPS per die, requiring even more communication between chips. Remember the bandwidth hierarchy: on-chip SRAM runs at tens of TB/s, HBM at single-digit TB/s, NVLink within a 72-GPU domain at high hundreds of GB/s, and between racks at low hundreds of GB/s. Each boundary is roughly a 10x drop. With less compute per chip, you need more chips to run the same model - which means more communication across those slower pipes that are already bottlenecking you.
3:31

If we get AGI before 2030 but can only produce ~200 GW a year of chips on advanced process nodes, can we ease the bottleneck by going back to older nodes? Some of the most singularity-pilled investors think so. Most of the FLOPS improvement between chip generations has come from numerics - not transistor density. But you could just design a new version of a 7nm chip like the A100 to have FP4. Holding numerics and die area constant, a B200 (2025 chip on 4nm) would only have ~3.5x more FLOPS than an A100 (2020 chip on 7nm). That seems like a manageable haircut - does it mean we can blow past the 200 GW ceiling? Dylan Patel explains why this is not an apples-to-apples comparison. If you run real-world performance analysis on DeepSeek V3 or Kimi K2.5, the performance gap between Hopper and Blackwell isn't 3x. It's ~20x. Why? Because inference is already memory-bandwidth-bound, and older chips have far fewer FLOPS per die, requiring even more communication between chips. Remember the bandwidth hierarchy: on-chip SRAM runs at tens of TB/s, HBM at single-digit TB/s, NVLink within a 72-GPU domain at high hundreds of GB/s, and between racks at low hundreds of GB/s. Each boundary is roughly a 10x drop. With less compute per chip, you need more chips to run the same model - which means more communication across those slower pipes that are already bottlenecking you.

Dwarkesh Patel

56,362 views • 4 months ago

🚨Governor DeSantis MOCKS the Florida GOP and their pathetic Chairman for RIGGING the primary and LYING to voters by pulling a BAIT AND SWITCH! Says SPECIAL INTERESTS are choosing our candidate! “They do this summit and they say we're gonna do a debate, a governor debate. All right, so people like signed up thinking that they'd have this debate and they said NO CANDIDATE CAN DO ANY OTHER DEBATE EXCEPT OUR DEBATE and then they say oh, well only ONE GUY can debate so we're not gonna have a debate! I mean, I think people feel that that's a BAIT-ND-SWITCH! The party's job is NOT to try to engineer an outcome. The Party's job is to inform Republican voters and and try to increase participation. I know when I was running in 2018, this is the fact, I mean most of the people in the RPOF were not for me in the primary. I mean, that's just the reality and so it is what it is. If you don't ever have debates then the only way to get known is basically the person that raises the most money from special interests. And then you spend money to do it and if you're not able to do that, then you don't even get known to begin with and that's challenging. And I know when I ran in 2018, I got outspent probably three to one and I think having certainly the June debate that we did was a huge, huge thing to be able to inform people about who I was, what my background was, all these other things. So I think there should be debates generally. I don't think the party has any right to control the debates but I think how this was handled, you've got to be on the up and up here and you say you could do debate, do a debate! If you're gonna say other candidates can't do any other debate, but our debate, well, you definitely got to do the debate, right? So it has not been I think done properly and look, I mean, voters can make decisions about how they think of all that.”
2:12

🚨Governor DeSantis MOCKS the Florida GOP and their pathetic Chairman for RIGGING the primary and LYING to voters by pulling a BAIT AND SWITCH! Says SPECIAL INTERESTS are choosing our candidate! “They do this summit and they say we're gonna do a debate, a governor debate. All right, so people like signed up thinking that they'd have this debate and they said NO CANDIDATE CAN DO ANY OTHER DEBATE EXCEPT OUR DEBATE and then they say oh, well only ONE GUY can debate so we're not gonna have a debate! I mean, I think people feel that that's a BAIT-ND-SWITCH! The party's job is NOT to try to engineer an outcome. The Party's job is to inform Republican voters and and try to increase participation. I know when I was running in 2018, this is the fact, I mean most of the people in the RPOF were not for me in the primary. I mean, that's just the reality and so it is what it is. If you don't ever have debates then the only way to get known is basically the person that raises the most money from special interests. And then you spend money to do it and if you're not able to do that, then you don't even get known to begin with and that's challenging. And I know when I ran in 2018, I got outspent probably three to one and I think having certainly the June debate that we did was a huge, huge thing to be able to inform people about who I was, what my background was, all these other things. So I think there should be debates generally. I don't think the party has any right to control the debates but I think how this was handled, you've got to be on the up and up here and you say you could do debate, do a debate! If you're gonna say other candidates can't do any other debate, but our debate, well, you definitely got to do the debate, right? So it has not been I think done properly and look, I mean, voters can make decisions about how they think of all that.”

Chris Nelson 🏝️🇺🇸

30,272 views • 1 month ago

Rogin: It looks very likely that we're about to restart bombing of Iran. And there are a lot of reports out there that this is about to happen. And what that means is that all of the problems, not just with the US economy, with the economy of every country in the world, are about to get worse. And it means that the Iran war is going to go much longer than otherwise anticipated. And that is a disaster not just for Americans, but for all people. And then if you think about the effects on energy and then manufacturing, and then rates will continue to go up, and then inflation, it's a it's a calamity for the world economy with no strategy whatsoever.
1:12

Rogin: It looks very likely that we're about to restart bombing of Iran. And there are a lot of reports out there that this is about to happen. And what that means is that all of the problems, not just with the US economy, with the economy of every country in the world, are about to get worse. And it means that the Iran war is going to go much longer than otherwise anticipated. And that is a disaster not just for Americans, but for all people. And then if you think about the effects on energy and then manufacturing, and then rates will continue to go up, and then inflation, it's a it's a calamity for the world economy with no strategy whatsoever.

Acyn

74,258 views • 3 months ago

ELON: TESLA FUTURE'S HINGES ON THE AI5 CHIP PROJECT "The AI5 chip, which is going to be a great chip, all of Tesla hinges on that chip. That's the chip that is going into our next generation of self-driving cars, and it's also essential for the Optimus robot. So that chip program was in bad shape. It wasn't closing because it's quite an ambitious chip design, and it really wasn't on a path to success. And then we also had the Dojo program, which was doing okay but not on a path to be competitive with NVIDIA. So I collapsed the two programs into just one program, which is to get everyone focused on the AI5 chip, which is essential. Continue to use NVIDIA for training, but we need the AI5 inference chip, which is a very powerful chip, but it's also a very low-power chip. So it doesn't use a lot of power. Its performance per watt is extremely good. I think it's probably going to be at least 2 to 4 times better than NVIDIA performance per watt at the inference level in the car and the robot. And I don't know, 10% of the cost of an NVIDIA chip, something like that. So these are very important numbers to achieve." Source: Elon Musk, Baron Capital, Ron Baron, Tesla
2:09

ELON: TESLA FUTURE'S HINGES ON THE AI5 CHIP PROJECT "The AI5 chip, which is going to be a great chip, all of Tesla hinges on that chip. That's the chip that is going into our next generation of self-driving cars, and it's also essential for the Optimus robot. So that chip program was in bad shape. It wasn't closing because it's quite an ambitious chip design, and it really wasn't on a path to success. And then we also had the Dojo program, which was doing okay but not on a path to be competitive with NVIDIA. So I collapsed the two programs into just one program, which is to get everyone focused on the AI5 chip, which is essential. Continue to use NVIDIA for training, but we need the AI5 inference chip, which is a very powerful chip, but it's also a very low-power chip. So it doesn't use a lot of power. Its performance per watt is extremely good. I think it's probably going to be at least 2 to 4 times better than NVIDIA performance per watt at the inference level in the car and the robot. And I don't know, 10% of the cost of an NVIDIA chip, something like that. So these are very important numbers to achieve." Source: Elon Musk, Baron Capital, Ron Baron, Tesla

Mario Nawfal

225,142 views • 9 months ago

"My view is that you're putting the cart before the horse if you think that somehow you can have a bunch of policy points and nicely drafted bills, that somehow you're going to offer up to fix all these problems when Donald J. Trump is president. He's not going to sign them. ... I think it's kind of weird that people are saying, 'Well let's talk about housing and all of that,' ignoring what is standing in the way of getting all of that done, which is this criminal. And that's the point of my campaign—to show people you can't do one without the other."
1:28

"My view is that you're putting the cart before the horse if you think that somehow you can have a bunch of policy points and nicely drafted bills, that somehow you're going to offer up to fix all these problems when Donald J. Trump is president. He's not going to sign them. ... I think it's kind of weird that people are saying, 'Well let's talk about housing and all of that,' ignoring what is standing in the way of getting all of that done, which is this criminal. And that's the point of my campaign—to show people you can't do one without the other."

George Conway ⚖️🇺🇸

108,842 views • 2 months ago