正在加载视频...

视频加载失败

返回首页

Gavin Baker on the one variable that now dominates all others in AI compute: "the more memory you put with flop for a given unit of compute, the more tokens you get out -- it's the single most important thing you could do to increase token output per unit...

firesidealpha's profile picture

Fireside Alpha

11,946 subscribers

18,521 次观看 • 1 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Gavin Baker says compute players could risk their entire business if they break memory LTAs "Let's just think about the game theory of breaking an LTA. So there's four companies that matter at scale. There's Amazon with their Trainiums, there's Google with their TPUs, there's AMD, and then there's Nvidia, who's much bigger than everybody else combined." "The more memory you put with FLOPs for a given unit of compute, the more tokens you get out. It's the single most important thing you could do to increase token output per unit of compute." "I think for the next several years market share is going to be determined by supply chain allocations and what you have pre purchased." "If you break your LTA and then in the next two or three years for any reason leverage shifts back to the memory guys, you're out of business. It's over." "If you break an LTA, they just say, 'OK, fine. You know what? Great, you broke the price agreement. We're going to break the volume agreement, and screw you, we're going to give the volume to your competitor.' You just lost share."
3:27

Gavin Baker says compute players could risk their entire business if they break memory LTAs "Let's just think about the game theory of breaking an LTA. So there's four companies that matter at scale. There's Amazon with their Trainiums, there's Google with their TPUs, there's AMD, and then there's Nvidia, who's much bigger than everybody else combined." "The more memory you put with FLOPs for a given unit of compute, the more tokens you get out. It's the single most important thing you could do to increase token output per unit of compute." "I think for the next several years market share is going to be determined by supply chain allocations and what you have pre purchased." "If you break your LTA and then in the next two or three years for any reason leverage shifts back to the memory guys, you're out of business. It's over." "If you break an LTA, they just say, 'OK, fine. You know what? Great, you broke the price agreement. We're going to break the volume agreement, and screw you, we're going to give the volume to your competitor.' You just lost share."

dnap

47,877 次观看 • 1 个月前

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."
0:52

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."

Fireside Alpha

66,227 次观看 • 3 个月前

BREAKING: Dan Ives says SpaceX could become one of the best AI investments in the market. "When you look at it on the compute side to space, it's much more of an AI play. When you go out in the next 2-3 years, if they execute, I could argue this becomes one of the best AI plays in the market and that's why we are bullish."
0:25

BREAKING: Dan Ives says SpaceX could become one of the best AI investments in the market. "When you look at it on the compute side to space, it's much more of an AI play. When you go out in the next 2-3 years, if they execute, I could argue this becomes one of the best AI plays in the market and that's why we are bullish."

DogeDesigner

86,983 次观看 • 2 个月前

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

130,756 次观看 • 2 个月前

Someone bought a Mac Mini for local AI. 64GB of unified memory seemed like the obvious choice. More memory. Bigger models. No VRAM limits. Then they put it against an RTX 3060. And things got weird. The Mac Mini M4 Pro pushed Qwen 2.5 at 38 tok/s. The old RTX 3060 with just 12GB VRAM hit 52 tok/s. That’s the interesting part about local AI. Apple and NVIDIA are solving different problems. Apple gives you a huge pool of memory. NVIDIA gives you dedicated compute and CUDA. One helps you fit bigger models. The other can run smaller models faster. Memory gets the model in. Compute gets the tokens out.
1:11

Someone bought a Mac Mini for local AI. 64GB of unified memory seemed like the obvious choice. More memory. Bigger models. No VRAM limits. Then they put it against an RTX 3060. And things got weird. The Mac Mini M4 Pro pushed Qwen 2.5 at 38 tok/s. The old RTX 3060 with just 12GB VRAM hit 52 tok/s. That’s the interesting part about local AI. Apple and NVIDIA are solving different problems. Apple gives you a huge pool of memory. NVIDIA gives you dedicated compute and CUDA. One helps you fit bigger models. The other can run smaller models faster. Memory gets the model in. Compute gets the tokens out.

KyzoroX

26,559 次观看 • 1 个月前

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.
0:50

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 次观看 • 2 个月前

Jaylen Brown says players should be able to own equity in teams because they help increase the value of NBA franchises “Players should be able to invest alongside ownership groups and business opportunities. I don't understand why that’s ever been a thing. It's like you’re an athlete and they make it seem like they can control how much wealth or growth that you could actually accumulate. I think that's wrong” “I also think that any other major corporation, if you work for Apple, Nike or anywhere, if you're a CEO, if you're someone who has been on a board for a large amount of time, you get equity in a company at some point. You are part of it” “I think athletes should be looked at in the same way. You played for the Celtics for 20 years, you should get a piece of equity because you helped accumulate the growth. That's the part that gets lost in translation. The sweat equity you put in. You get compensated for doing your job, but you don't get compensated for the growth. At major corporations and companies you do, or at least some of the big ones”
1:18

Jaylen Brown says players should be able to own equity in teams because they help increase the value of NBA franchises “Players should be able to invest alongside ownership groups and business opportunities. I don't understand why that’s ever been a thing. It's like you’re an athlete and they make it seem like they can control how much wealth or growth that you could actually accumulate. I think that's wrong” “I also think that any other major corporation, if you work for Apple, Nike or anywhere, if you're a CEO, if you're someone who has been on a board for a large amount of time, you get equity in a company at some point. You are part of it” “I think athletes should be looked at in the same way. You played for the Celtics for 20 years, you should get a piece of equity because you helped accumulate the growth. That's the part that gets lost in translation. The sweat equity you put in. You get compensated for doing your job, but you don't get compensated for the growth. At major corporations and companies you do, or at least some of the big ones”

dank

3,039,759 次观看 • 2 个月前

Gavin Baker: "if you're a foundation model company and you do not have unique data and internet scale distribution, you are the fastest depreciating asset in human history - I think most of these companies are zeros, and there's like 10 of them" this is him explaining why the 100 largest buildings in the world will be data centers, why there are only three places left in America you can legally build one, and what he says is setting up the first real memory cycle since 1996 "in our lifetime there will be data centers that are bigger than Manhattan" "the two rate limiting factors for AI are no longer compute and GPUs - it's power, finding places to plug the GPUs in" "there's only three places in the United States you can put one of these - it's got to be near multiple nuclear reactors that are unregulated within 50 miles of each other" "if Apple doubles the memory in the iPhone and Google does the same for Android at the same time, you could have the first real memory cycle since 1996 - and in 1996 Micron was a 50 bagger" "the only hope most of these companies have is for Amazon to acquire them the way Microsoft did with Inflection" bookmark & watch the full conversation ↓
11:48

Gavin Baker: "if you're a foundation model company and you do not have unique data and internet scale distribution, you are the fastest depreciating asset in human history - I think most of these companies are zeros, and there's like 10 of them" this is him explaining why the 100 largest buildings in the world will be data centers, why there are only three places left in America you can legally build one, and what he says is setting up the first real memory cycle since 1996 "in our lifetime there will be data centers that are bigger than Manhattan" "the two rate limiting factors for AI are no longer compute and GPUs - it's power, finding places to plug the GPUs in" "there's only three places in the United States you can put one of these - it's got to be near multiple nuclear reactors that are unregulated within 50 miles of each other" "if Apple doubles the memory in the iPhone and Google does the same for Android at the same time, you could have the first real memory cycle since 1996 - and in 1996 Micron was a 50 bagger" "the only hope most of these companies have is for Amazon to acquire them the way Microsoft did with Inflection" bookmark & watch the full conversation ↓

Сarm1ne

274,201 次观看 • 1 个月前

Bloomberg's Joe Weisenthal observes the scary double bind that compute prices 10xing puts hyperscalers in: "One of the stylized facts in recent years has been the extraordinary drop in token prices." "If you go back to GPT-3, token prices are down like 99.9%." "We think there's some iron law that token prices will get cheaper over time." "But maybe not. We thought there was an iron law that memory prices would drop over time. Or that any sort of hardware would drop over time." "But the story of the last two years is that that law can reverse." "It's not impossible to imagine that compute prices get significantly more expensive." "But that doesn't mean you can stop spending on it. Because you have to worry about hardening yourself for cybersecurity." "There's no, 'This is not economical for me anymore. I want to tap out of the whole AI race.'" "Well then, your competitor develops a stronger model. And then you're done. You're really done." "This is the scary thing. What if the situation is, you're bleeding free cash flow, but you also don't have the option to opt out of the game?"
2:00

Bloomberg's Joe Weisenthal observes the scary double bind that compute prices 10xing puts hyperscalers in: "One of the stylized facts in recent years has been the extraordinary drop in token prices." "If you go back to GPT-3, token prices are down like 99.9%." "We think there's some iron law that token prices will get cheaper over time." "But maybe not. We thought there was an iron law that memory prices would drop over time. Or that any sort of hardware would drop over time." "But the story of the last two years is that that law can reverse." "It's not impossible to imagine that compute prices get significantly more expensive." "But that doesn't mean you can stop spending on it. Because you have to worry about hardening yourself for cybersecurity." "There's no, 'This is not economical for me anymore. I want to tap out of the whole AI race.'" "Well then, your competitor develops a stronger model. And then you're done. You're really done." "This is the scary thing. What if the situation is, you're bleeding free cash flow, but you also don't have the option to opt out of the game?"

TBPN

92,708 次观看 • 2 个月前

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."
1:53

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."

Trade Whisperer

180,543 次观看 • 2 个月前

Ben Thompson says the memory makers are Iran, and by making the Strait of Hormuz mistake are inviting the whole industry to build around them "The other thing, the risk they run, is Apple. Apple's lobbying to get Chinese memory, right?" "And what is the number one focus of algorithmic changes? How can we use less memory?" "I think the memory makers probably screw themselves in the long run by creating such a massive target on their back." "I've analogized memory makers to Iran. The issue with the Strait of Hormuz is it's very effective. It's more effective if you don't use it, cuz then it's always hanging out there as something you could do. Now they did it. Turns out it worked." "But the UAE and Saudi Arabia, they're going to build pipelines. They're going to build new ports. They're not going to let this happen again." "It's very painful right now. But say Iran wants to close the Strait of Hormuz in 2035. It's not going to have any effect, because it will have been built around." "My concern for the memory makers is they might have done the same thing. No one's going to let themselves get in this situation again as far as memory goes."
0:58

Ben Thompson says the memory makers are Iran, and by making the Strait of Hormuz mistake are inviting the whole industry to build around them "The other thing, the risk they run, is Apple. Apple's lobbying to get Chinese memory, right?" "And what is the number one focus of algorithmic changes? How can we use less memory?" "I think the memory makers probably screw themselves in the long run by creating such a massive target on their back." "I've analogized memory makers to Iran. The issue with the Strait of Hormuz is it's very effective. It's more effective if you don't use it, cuz then it's always hanging out there as something you could do. Now they did it. Turns out it worked." "But the UAE and Saudi Arabia, they're going to build pipelines. They're going to build new ports. They're not going to let this happen again." "It's very painful right now. But say Iran wants to close the Strait of Hormuz in 2035. It's not going to have any effect, because it will have been built around." "My concern for the memory makers is they might have done the same thing. No one's going to let themselves get in this situation again as far as memory goes."

Fireside Alpha

157,042 次观看 • 1 个月前

The difference between memory and a second brain... In Hermes you have a memory.md file, but it's not particularly useful for storing a lot of info. Your second brain is a type of memory where you can distil down your raw inputs into insight you can use with AI. If you want to build a second brain with GPT6 Astra, check out the article I put together below.
0:44

The difference between memory and a second brain... In Hermes you have a memory.md file, but it's not particularly useful for storing a lot of info. Your second brain is a type of memory where you can distil down your raw inputs into insight you can use with AI. If you want to build a second brain with GPT6 Astra, check out the article I put together below.

Tom

24,164 次观看 • 17 天前

OpenClaw's memory out of the box sucks With this simple Obsidian system, it's actually awesome In this video I go over using Obsidian as a memory layer for your AI agent, making it SO much more smarter than it is now If you use OpenClaw or Hermes, you need to watch this:
10:57

OpenClaw's memory out of the box sucks With this simple Obsidian system, it's actually awesome In this video I go over using Obsidian as a memory layer for your AI agent, making it SO much more smarter than it is now If you use OpenClaw or Hermes, you need to watch this:

Alex Finn

89,897 次观看 • 5 个月前

Ben Thompson uses the Strait of Hormuz to explain why memory makers squeezing the market now guarantees they lose their leverage later: "I think the memory makers probably screw themselves in the long run by creating such a massive target on their back. The issue with the Strait of Hormuz is that it's very effective if you don't use it. Then it's always hanging out there as something you could do. Now they did, and it turns out it worked. But the UAE and Saudi Arabia, they're going to build pipelines, they're going to build new ports. They're not going to let this happen again. So it's very painful right now, but say Iran wants to close the Strait of Hormuz in 2035. It's not going to have any effect, because it will have been built around. My concern for the memory makers is they might have done the same thing. No one is going to let themselves get in this situation again."
1:31

Ben Thompson uses the Strait of Hormuz to explain why memory makers squeezing the market now guarantees they lose their leverage later: "I think the memory makers probably screw themselves in the long run by creating such a massive target on their back. The issue with the Strait of Hormuz is that it's very effective if you don't use it. Then it's always hanging out there as something you could do. Now they did, and it turns out it worked. But the UAE and Saudi Arabia, they're going to build pipelines, they're going to build new ports. They're not going to let this happen again. So it's very painful right now, but say Iran wants to close the Strait of Hormuz in 2035. It's not going to have any effect, because it will have been built around. My concern for the memory makers is they might have done the same thing. No one is going to let themselves get in this situation again."

Invest Like the Best

21,582 次观看 • 1 个月前

GAVIN BAKER: NVIDIA IS NO LONGER JUST A CHIP COMPANY. Gavin Baker recently laid out one of the sharpest observations about the current AI hardware cycle, and it explains why Nvidia's multiple looks strange relative to the strategic position the company actually occupies. The starting point is a shift in how semiconductor supply is being allocated. Historically, buyers with enormous volume like Apple could break long-term agreements with suppliers whenever they wanted. There was no meaningful consequence, because the supplier had nowhere else to go. That world is over. In the current cycle, there are at least four major memory buyers with genuine scale, plus a wave of AI startups adding to demand. The dynamic has flipped. If a hyperscaler breaks an LTA on price, the supplier can now retaliate by reallocating volume to a competitor. In a cyclical industry where oversupply is always followed by undersupply, breaking an LTA today can cost you your entire allocation the next time capacity gets tight. That leverage is one reason Nvidia's position is so unusual. Every constraint in the AI buildout tilts in its favor. Nothing on earth is more financeable than an Nvidia GPU. And on land and power, Nvidia is playing the matchmaking chess game between customers, energy suppliers, and infrastructure partners. The most interesting development Baker highlighted is Nvidia's new business model. He described it as a credit wrapper with a revenue share above a certain GPU price floor. Nvidia is financing customer deployments while retaining upside if GPU prices stay elevated. This converts Nvidia from a pure hardware vendor into something structurally closer to a cloud franchise via royalties on the compute being deployed on its chips. Nvidia could end up with an effectively massive cloud business very quickly, not through building data centers but through collecting royalties on the compute those data centers produce. The market is still pricing $NVDA as a semiconductor company. The company itself has already moved beyond that model. The gap between the two will eventually close. Invest Like the Best Patrick OShaughnessy Gavin Baker
1:37

GAVIN BAKER: NVIDIA IS NO LONGER JUST A CHIP COMPANY. Gavin Baker recently laid out one of the sharpest observations about the current AI hardware cycle, and it explains why Nvidia's multiple looks strange relative to the strategic position the company actually occupies. The starting point is a shift in how semiconductor supply is being allocated. Historically, buyers with enormous volume like Apple could break long-term agreements with suppliers whenever they wanted. There was no meaningful consequence, because the supplier had nowhere else to go. That world is over. In the current cycle, there are at least four major memory buyers with genuine scale, plus a wave of AI startups adding to demand. The dynamic has flipped. If a hyperscaler breaks an LTA on price, the supplier can now retaliate by reallocating volume to a competitor. In a cyclical industry where oversupply is always followed by undersupply, breaking an LTA today can cost you your entire allocation the next time capacity gets tight. That leverage is one reason Nvidia's position is so unusual. Every constraint in the AI buildout tilts in its favor. Nothing on earth is more financeable than an Nvidia GPU. And on land and power, Nvidia is playing the matchmaking chess game between customers, energy suppliers, and infrastructure partners. The most interesting development Baker highlighted is Nvidia's new business model. He described it as a credit wrapper with a revenue share above a certain GPU price floor. Nvidia is financing customer deployments while retaining upside if GPU prices stay elevated. This converts Nvidia from a pure hardware vendor into something structurally closer to a cloud franchise via royalties on the compute being deployed on its chips. Nvidia could end up with an effectively massive cloud business very quickly, not through building data centers but through collecting royalties on the compute those data centers produce. The market is still pricing $NVDA as a semiconductor company. The company itself has already moved beyond that model. The gap between the two will eventually close. Invest Like the Best Patrick OShaughnessy Gavin Baker

Lumida Wealth Management

194,482 次观看 • 1 个月前

Nvidia will pay you to put a $250,000 data center outside your house you don't touch it, you don't manage it, you just collect around $1,000 a month for the electricity and Wi-Fi it uses it looks like an AC unit, it's actually 16 Blackwell GPUs running AI workloads around the clock it's built by Span, a startup Nvidia invested in to make distributed compute a reality the reason Nvidia put money into this is simple they need compute everywhere and building data centers takes years and their solution is to skip the construction entirely, use residential grid capacity and deploy through your neighborhood instead this is much cheaper, faster, and can scale to millions of homes but there is still a question nobody has answered yet, what happens if someone steals it also, technically they could give each homeowner access to a slice of the compute even 2% of 16 Blackwell GPUs is more processing power than most people will ever use bookmark this, it's worth coming back to when you have time
0:57

Nvidia will pay you to put a $250,000 data center outside your house you don't touch it, you don't manage it, you just collect around $1,000 a month for the electricity and Wi-Fi it uses it looks like an AC unit, it's actually 16 Blackwell GPUs running AI workloads around the clock it's built by Span, a startup Nvidia invested in to make distributed compute a reality the reason Nvidia put money into this is simple they need compute everywhere and building data centers takes years and their solution is to skip the construction entirely, use residential grid capacity and deploy through your neighborhood instead this is much cheaper, faster, and can scale to millions of homes but there is still a question nobody has answered yet, what happens if someone steals it also, technically they could give each homeowner access to a slice of the compute even 2% of 16 Blackwell GPUs is more processing power than most people will ever use bookmark this, it's worth coming back to when you have time

Anatoli Kopadze

403,793 次观看 • 4 个月前

TESLA ISN'T JUST AHEAD IN AI, IT'S AN ORDER OF MAGNITUDE MORE EFFICIENT THAN THE REST OF THE WORLD "I actually think Tesla is ahead of the rest of the world in intelligence density of AI by an order of magnitude or more This is going to sound like a pretty bold statement, but I know what the intelligence efficiency of the big models are like, Grok and a bunch of the other models Tesla’s AI is, in terms of its memory efficiency, I think more than an order of magnitude better. The Tesla AI is very compute-efficient and very memory-efficient One of the metrics one should consider for any given AI model is the intelligence per gigabyte. Especially when you’re constrained on RAM, having an AI that has very high intelligence density per gigabyte for a given number of gigabytes, how much functionality can you get out of it" - Elon Musk
0:59

TESLA ISN'T JUST AHEAD IN AI, IT'S AN ORDER OF MAGNITUDE MORE EFFICIENT THAN THE REST OF THE WORLD "I actually think Tesla is ahead of the rest of the world in intelligence density of AI by an order of magnitude or more This is going to sound like a pretty bold statement, but I know what the intelligence efficiency of the big models are like, Grok and a bunch of the other models Tesla’s AI is, in terms of its memory efficiency, I think more than an order of magnitude better. The Tesla AI is very compute-efficient and very memory-efficient One of the metrics one should consider for any given AI model is the intelligence per gigabyte. Especially when you’re constrained on RAM, having an AI that has very high intelligence density per gigabyte for a given number of gigabytes, how much functionality can you get out of it" - Elon Musk

X Freeze

62,736 次观看 • 8 个月前

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.
3:31

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.

Karl Mehta

16,619 次观看 • 3 个月前

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.
0:58

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.

Ricardo

594,643 次观看 • 8 个月前

Atreides' Gavin Baker says that once Starship is reusable, the economics of orbital compute crush the economics of terrestrial compute. He lays out the math: "Once $SPCX can reuse Starship, the math for orbital compute becomes pretty compelling." "It's $60B to bring on a GW terrestrially. $25B is power and cooling — you don't need that in space. And so the right comp for that $35B in IT equipment — GPU, CPU, switches, memory, storage — is that $25B [in power and cooling] vs. the cost of launch." "And once Starship is reusable, I think the cost to launch is $5B. So that means you can put a GW into space for [$40B]. The GW on earth is $60B."
0:53

Atreides' Gavin Baker says that once Starship is reusable, the economics of orbital compute crush the economics of terrestrial compute. He lays out the math: "Once $SPCX can reuse Starship, the math for orbital compute becomes pretty compelling." "It's $60B to bring on a GW terrestrially. $25B is power and cooling — you don't need that in space. And so the right comp for that $35B in IT equipment — GPU, CPU, switches, memory, storage — is that $25B [in power and cooling] vs. the cost of launch." "And once Starship is reusable, I think the cost to launch is $5B. So that means you can put a GW into space for [$40B]. The GW on earth is $60B."

TBPN

104,010 次观看 • 3 个月前