Loading video...

Video Failed to Load

Go Home

Gavin Baker on the one variable that now dominates all others in AI compute: "the more memory you put with flop for a given unit of compute, the more tokens you get out -- it's the single most important thing you could do to increase token output per unit...

firesidealpha's profile picture

Fireside Alpha

11,946 subscribers

18,521 views • 4 days ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Gavin Baker says compute players could risk their entire business if they break memory LTAs "Let's just think about the game theory of breaking an LTA. So there's four companies that matter at scale. There's Amazon with their Trainiums, there's Google with their TPUs, there's AMD, and then there's Nvidia, who's much bigger than everybody else combined." "The more memory you put with FLOPs for a given unit of compute, the more tokens you get out. It's the single most important thing you could do to increase token output per unit of compute." "I think for the next several years market share is going to be determined by supply chain allocations and what you have pre purchased." "If you break your LTA and then in the next two or three years for any reason leverage shifts back to the memory guys, you're out of business. It's over." "If you break an LTA, they just say, 'OK, fine. You know what? Great, you broke the price agreement. We're going to break the volume agreement, and screw you, we're going to give the volume to your competitor.' You just lost share."
3:27

Gavin Baker says compute players could risk their entire business if they break memory LTAs "Let's just think about the game theory of breaking an LTA. So there's four companies that matter at scale. There's Amazon with their Trainiums, there's Google with their TPUs, there's AMD, and then there's Nvidia, who's much bigger than everybody else combined." "The more memory you put with FLOPs for a given unit of compute, the more tokens you get out. It's the single most important thing you could do to increase token output per unit of compute." "I think for the next several years market share is going to be determined by supply chain allocations and what you have pre purchased." "If you break your LTA and then in the next two or three years for any reason leverage shifts back to the memory guys, you're out of business. It's over." "If you break an LTA, they just say, 'OK, fine. You know what? Great, you broke the price agreement. We're going to break the volume agreement, and screw you, we're going to give the volume to your competitor.' You just lost share."

dnap

46,835 views • 4 days ago

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."
0:52

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."

Fireside Alpha

66,227 views • 1 month ago

BREAKING: Dan Ives says SpaceX could become one of the best AI investments in the market. "When you look at it on the compute side to space, it's much more of an AI play. When you go out in the next 2-3 years, if they execute, I could argue this becomes one of the best AI plays in the market and that's why we are bullish."
0:25

BREAKING: Dan Ives says SpaceX could become one of the best AI investments in the market. "When you look at it on the compute side to space, it's much more of an AI play. When you go out in the next 2-3 years, if they execute, I could argue this becomes one of the best AI plays in the market and that's why we are bullish."

DogeDesigner

86,983 views • 1 month ago

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,678 views • 1 month ago

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.
0:50

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 views • 1 month ago

Jaylen Brown says players should be able to own equity in teams because they help increase the value of NBA franchises “Players should be able to invest alongside ownership groups and business opportunities. I don't understand why that’s ever been a thing. It's like you’re an athlete and they make it seem like they can control how much wealth or growth that you could actually accumulate. I think that's wrong” “I also think that any other major corporation, if you work for Apple, Nike or anywhere, if you're a CEO, if you're someone who has been on a board for a large amount of time, you get equity in a company at some point. You are part of it” “I think athletes should be looked at in the same way. You played for the Celtics for 20 years, you should get a piece of equity because you helped accumulate the growth. That's the part that gets lost in translation. The sweat equity you put in. You get compensated for doing your job, but you don't get compensated for the growth. At major corporations and companies you do, or at least some of the big ones”
1:18

Jaylen Brown says players should be able to own equity in teams because they help increase the value of NBA franchises “Players should be able to invest alongside ownership groups and business opportunities. I don't understand why that’s ever been a thing. It's like you’re an athlete and they make it seem like they can control how much wealth or growth that you could actually accumulate. I think that's wrong” “I also think that any other major corporation, if you work for Apple, Nike or anywhere, if you're a CEO, if you're someone who has been on a board for a large amount of time, you get equity in a company at some point. You are part of it” “I think athletes should be looked at in the same way. You played for the Celtics for 20 years, you should get a piece of equity because you helped accumulate the growth. That's the part that gets lost in translation. The sweat equity you put in. You get compensated for doing your job, but you don't get compensated for the growth. At major corporations and companies you do, or at least some of the big ones”

dank

3,034,757 views • 28 days ago

Gavin Baker: "if you're a foundation model company and you do not have unique data and internet scale distribution, you are the fastest depreciating asset in human history - I think most of these companies are zeros, and there's like 10 of them" this is him explaining why the 100 largest buildings in the world will be data centers, why there are only three places left in America you can legally build one, and what he says is setting up the first real memory cycle since 1996 "in our lifetime there will be data centers that are bigger than Manhattan" "the two rate limiting factors for AI are no longer compute and GPUs - it's power, finding places to plug the GPUs in" "there's only three places in the United States you can put one of these - it's got to be near multiple nuclear reactors that are unregulated within 50 miles of each other" "if Apple doubles the memory in the iPhone and Google does the same for Android at the same time, you could have the first real memory cycle since 1996 - and in 1996 Micron was a 50 bagger" "the only hope most of these companies have is for Amazon to acquire them the way Microsoft did with Inflection" bookmark & watch the full conversation ↓
11:48

Gavin Baker: "if you're a foundation model company and you do not have unique data and internet scale distribution, you are the fastest depreciating asset in human history - I think most of these companies are zeros, and there's like 10 of them" this is him explaining why the 100 largest buildings in the world will be data centers, why there are only three places left in America you can legally build one, and what he says is setting up the first real memory cycle since 1996 "in our lifetime there will be data centers that are bigger than Manhattan" "the two rate limiting factors for AI are no longer compute and GPUs - it's power, finding places to plug the GPUs in" "there's only three places in the United States you can put one of these - it's got to be near multiple nuclear reactors that are unregulated within 50 miles of each other" "if Apple doubles the memory in the iPhone and Google does the same for Android at the same time, you could have the first real memory cycle since 1996 - and in 1996 Micron was a 50 bagger" "the only hope most of these companies have is for Amazon to acquire them the way Microsoft did with Inflection" bookmark & watch the full conversation ↓

Сarm1ne

270,697 views • 6 days ago

Bloomberg's Joe Weisenthal observes the scary double bind that compute prices 10xing puts hyperscalers in: "One of the stylized facts in recent years has been the extraordinary drop in token prices." "If you go back to GPT-3, token prices are down like 99.9%." "We think there's some iron law that token prices will get cheaper over time." "But maybe not. We thought there was an iron law that memory prices would drop over time. Or that any sort of hardware would drop over time." "But the story of the last two years is that that law can reverse." "It's not impossible to imagine that compute prices get significantly more expensive." "But that doesn't mean you can stop spending on it. Because you have to worry about hardening yourself for cybersecurity." "There's no, 'This is not economical for me anymore. I want to tap out of the whole AI race.'" "Well then, your competitor develops a stronger model. And then you're done. You're really done." "This is the scary thing. What if the situation is, you're bleeding free cash flow, but you also don't have the option to opt out of the game?"
2:00

Bloomberg's Joe Weisenthal observes the scary double bind that compute prices 10xing puts hyperscalers in: "One of the stylized facts in recent years has been the extraordinary drop in token prices." "If you go back to GPT-3, token prices are down like 99.9%." "We think there's some iron law that token prices will get cheaper over time." "But maybe not. We thought there was an iron law that memory prices would drop over time. Or that any sort of hardware would drop over time." "But the story of the last two years is that that law can reverse." "It's not impossible to imagine that compute prices get significantly more expensive." "But that doesn't mean you can stop spending on it. Because you have to worry about hardening yourself for cybersecurity." "There's no, 'This is not economical for me anymore. I want to tap out of the whole AI race.'" "Well then, your competitor develops a stronger model. And then you're done. You're really done." "This is the scary thing. What if the situation is, you're bleeding free cash flow, but you also don't have the option to opt out of the game?"

TBPN

92,708 views • 10 days ago

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."
1:53

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."

Trade Whisperer

179,812 views • 16 days ago

OpenClaw's memory out of the box sucks With this simple Obsidian system, it's actually awesome In this video I go over using Obsidian as a memory layer for your AI agent, making it SO much more smarter than it is now If you use OpenClaw or Hermes, you need to watch this:
10:57

OpenClaw's memory out of the box sucks With this simple Obsidian system, it's actually awesome In this video I go over using Obsidian as a memory layer for your AI agent, making it SO much more smarter than it is now If you use OpenClaw or Hermes, you need to watch this:

Alex Finn

89,742 views • 4 months ago

Nvidia will pay you to put a $250,000 data center outside your house you don't touch it, you don't manage it, you just collect around $1,000 a month for the electricity and Wi-Fi it uses it looks like an AC unit, it's actually 16 Blackwell GPUs running AI workloads around the clock it's built by Span, a startup Nvidia invested in to make distributed compute a reality the reason Nvidia put money into this is simple they need compute everywhere and building data centers takes years and their solution is to skip the construction entirely, use residential grid capacity and deploy through your neighborhood instead this is much cheaper, faster, and can scale to millions of homes but there is still a question nobody has answered yet, what happens if someone steals it also, technically they could give each homeowner access to a slice of the compute even 2% of 16 Blackwell GPUs is more processing power than most people will ever use bookmark this, it's worth coming back to when you have time
0:57

Nvidia will pay you to put a $250,000 data center outside your house you don't touch it, you don't manage it, you just collect around $1,000 a month for the electricity and Wi-Fi it uses it looks like an AC unit, it's actually 16 Blackwell GPUs running AI workloads around the clock it's built by Span, a startup Nvidia invested in to make distributed compute a reality the reason Nvidia put money into this is simple they need compute everywhere and building data centers takes years and their solution is to skip the construction entirely, use residential grid capacity and deploy through your neighborhood instead this is much cheaper, faster, and can scale to millions of homes but there is still a question nobody has answered yet, what happens if someone steals it also, technically they could give each homeowner access to a slice of the compute even 2% of 16 Blackwell GPUs is more processing power than most people will ever use bookmark this, it's worth coming back to when you have time

Anatoli Kopadze

403,338 views • 2 months ago

GAVIN BAKER: NVIDIA IS NO LONGER JUST A CHIP COMPANY. Gavin Baker recently laid out one of the sharpest observations about the current AI hardware cycle, and it explains why Nvidia's multiple looks strange relative to the strategic position the company actually occupies. The starting point is a shift in how semiconductor supply is being allocated. Historically, buyers with enormous volume like Apple could break long-term agreements with suppliers whenever they wanted. There was no meaningful consequence, because the supplier had nowhere else to go. That world is over. In the current cycle, there are at least four major memory buyers with genuine scale, plus a wave of AI startups adding to demand. The dynamic has flipped. If a hyperscaler breaks an LTA on price, the supplier can now retaliate by reallocating volume to a competitor. In a cyclical industry where oversupply is always followed by undersupply, breaking an LTA today can cost you your entire allocation the next time capacity gets tight. That leverage is one reason Nvidia's position is so unusual. Every constraint in the AI buildout tilts in its favor. Nothing on earth is more financeable than an Nvidia GPU. And on land and power, Nvidia is playing the matchmaking chess game between customers, energy suppliers, and infrastructure partners. The most interesting development Baker highlighted is Nvidia's new business model. He described it as a credit wrapper with a revenue share above a certain GPU price floor. Nvidia is financing customer deployments while retaining upside if GPU prices stay elevated. This converts Nvidia from a pure hardware vendor into something structurally closer to a cloud franchise via royalties on the compute being deployed on its chips. Nvidia could end up with an effectively massive cloud business very quickly, not through building data centers but through collecting royalties on the compute those data centers produce. The market is still pricing $NVDA as a semiconductor company. The company itself has already moved beyond that model. The gap between the two will eventually close. Invest Like the Best Patrick OShaughnessy Gavin Baker
1:37

GAVIN BAKER: NVIDIA IS NO LONGER JUST A CHIP COMPANY. Gavin Baker recently laid out one of the sharpest observations about the current AI hardware cycle, and it explains why Nvidia's multiple looks strange relative to the strategic position the company actually occupies. The starting point is a shift in how semiconductor supply is being allocated. Historically, buyers with enormous volume like Apple could break long-term agreements with suppliers whenever they wanted. There was no meaningful consequence, because the supplier had nowhere else to go. That world is over. In the current cycle, there are at least four major memory buyers with genuine scale, plus a wave of AI startups adding to demand. The dynamic has flipped. If a hyperscaler breaks an LTA on price, the supplier can now retaliate by reallocating volume to a competitor. In a cyclical industry where oversupply is always followed by undersupply, breaking an LTA today can cost you your entire allocation the next time capacity gets tight. That leverage is one reason Nvidia's position is so unusual. Every constraint in the AI buildout tilts in its favor. Nothing on earth is more financeable than an Nvidia GPU. And on land and power, Nvidia is playing the matchmaking chess game between customers, energy suppliers, and infrastructure partners. The most interesting development Baker highlighted is Nvidia's new business model. He described it as a credit wrapper with a revenue share above a certain GPU price floor. Nvidia is financing customer deployments while retaining upside if GPU prices stay elevated. This converts Nvidia from a pure hardware vendor into something structurally closer to a cloud franchise via royalties on the compute being deployed on its chips. Nvidia could end up with an effectively massive cloud business very quickly, not through building data centers but through collecting royalties on the compute those data centers produce. The market is still pricing $NVDA as a semiconductor company. The company itself has already moved beyond that model. The gap between the two will eventually close. Invest Like the Best Patrick OShaughnessy Gavin Baker

Lumida Wealth Management

184,590 views • 4 days ago

TESLA ISN'T JUST AHEAD IN AI, IT'S AN ORDER OF MAGNITUDE MORE EFFICIENT THAN THE REST OF THE WORLD "I actually think Tesla is ahead of the rest of the world in intelligence density of AI by an order of magnitude or more This is going to sound like a pretty bold statement, but I know what the intelligence efficiency of the big models are like, Grok and a bunch of the other models Tesla’s AI is, in terms of its memory efficiency, I think more than an order of magnitude better. The Tesla AI is very compute-efficient and very memory-efficient One of the metrics one should consider for any given AI model is the intelligence per gigabyte. Especially when you’re constrained on RAM, having an AI that has very high intelligence density per gigabyte for a given number of gigabytes, how much functionality can you get out of it" - Elon Musk
0:59

TESLA ISN'T JUST AHEAD IN AI, IT'S AN ORDER OF MAGNITUDE MORE EFFICIENT THAN THE REST OF THE WORLD "I actually think Tesla is ahead of the rest of the world in intelligence density of AI by an order of magnitude or more This is going to sound like a pretty bold statement, but I know what the intelligence efficiency of the big models are like, Grok and a bunch of the other models Tesla’s AI is, in terms of its memory efficiency, I think more than an order of magnitude better. The Tesla AI is very compute-efficient and very memory-efficient One of the metrics one should consider for any given AI model is the intelligence per gigabyte. Especially when you’re constrained on RAM, having an AI that has very high intelligence density per gigabyte for a given number of gigabytes, how much functionality can you get out of it" - Elon Musk

X Freeze

62,736 views • 6 months ago

Atreides' Gavin Baker says that once Starship is reusable, the economics of orbital compute crush the economics of terrestrial compute. He lays out the math: "Once $SPCX can reuse Starship, the math for orbital compute becomes pretty compelling." "It's $60B to bring on a GW terrestrially. $25B is power and cooling — you don't need that in space. And so the right comp for that $35B in IT equipment — GPU, CPU, switches, memory, storage — is that $25B [in power and cooling] vs. the cost of launch." "And once Starship is reusable, I think the cost to launch is $5B. So that means you can put a GW into space for [$40B]. The GW on earth is $60B."
0:53

Atreides' Gavin Baker says that once Starship is reusable, the economics of orbital compute crush the economics of terrestrial compute. He lays out the math: "Once $SPCX can reuse Starship, the math for orbital compute becomes pretty compelling." "It's $60B to bring on a GW terrestrially. $25B is power and cooling — you don't need that in space. And so the right comp for that $35B in IT equipment — GPU, CPU, switches, memory, storage — is that $25B [in power and cooling] vs. the cost of launch." "And once Starship is reusable, I think the cost to launch is $5B. So that means you can put a GW into space for [$40B]. The GW on earth is $60B."

TBPN

102,778 views • 1 month ago

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.
3:31

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.

Karl Mehta

16,619 views • 1 month ago

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.
0:58

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.

Ricardo

594,643 views • 6 months ago

Jensen Huang just replaced the most important metric in global economics. Not trade volume. Not oil output. Not manufacturing. Compute. Huang: “Compute equals GDP. I know that for certain.” He did not say probably. He said certain. If your nation does not produce compute, it does not produce intelligence. If it does not produce intelligence, it does not produce revenue. Two links in the chain. Miss one and the whole thing breaks. Huang: “Not one country in the future will say, ‘Guess what, we’re gonna opt out on intelligence.’” Because opting out of compute is not a strategic decision. It is an extinction schedule. Every country that does not build its own inference capacity becomes a tenant in someone else’s infrastructure. Not an ally. Not a partner. A dependent. And dependents do not negotiate terms. They accept them. But this is not just a story about nations. Huang: “The entire software industry will be token-driven.” Every product. Every platform. Every service you touch. The entire business model of software is about to be measured in tokens consumed. Not seats sold. Not licenses renewed. Tokens burned. Software used to be a thing you bought. Now it is a thing that thinks. And thinking costs compute. Every query. Every action. Every decision the machine makes on your behalf. The meter is always running. Huang: “The entire internet industry could take 100% of their CapEx and make it AI because it’s better.” Not ten percent. Not a pilot program. One hundred percent. The moment any internet service rebuilds itself on generative intelligence, it outperforms every version that came before it. Search. Ads. Recommendation. Infrastructure. All of it. Better on contact. CapEx follows. All of it. Trillions moving in one direction with no offramp. The companies still budgeting AI as a line item are telling you exactly how much they understand. AI is not the line item. AI is the budget. The global economy is being re-denominated in a currency most people have not even heard of yet. Tokens. Whoever controls the supply of that currency is not playing in the new economy. They are the house. And the house does not lose.
2:07

Jensen Huang just replaced the most important metric in global economics. Not trade volume. Not oil output. Not manufacturing. Compute. Huang: “Compute equals GDP. I know that for certain.” He did not say probably. He said certain. If your nation does not produce compute, it does not produce intelligence. If it does not produce intelligence, it does not produce revenue. Two links in the chain. Miss one and the whole thing breaks. Huang: “Not one country in the future will say, ‘Guess what, we’re gonna opt out on intelligence.’” Because opting out of compute is not a strategic decision. It is an extinction schedule. Every country that does not build its own inference capacity becomes a tenant in someone else’s infrastructure. Not an ally. Not a partner. A dependent. And dependents do not negotiate terms. They accept them. But this is not just a story about nations. Huang: “The entire software industry will be token-driven.” Every product. Every platform. Every service you touch. The entire business model of software is about to be measured in tokens consumed. Not seats sold. Not licenses renewed. Tokens burned. Software used to be a thing you bought. Now it is a thing that thinks. And thinking costs compute. Every query. Every action. Every decision the machine makes on your behalf. The meter is always running. Huang: “The entire internet industry could take 100% of their CapEx and make it AI because it’s better.” Not ten percent. Not a pilot program. One hundred percent. The moment any internet service rebuilds itself on generative intelligence, it outperforms every version that came before it. Search. Ads. Recommendation. Infrastructure. All of it. Better on contact. CapEx follows. All of it. Trillions moving in one direction with no offramp. The companies still budgeting AI as a line item are telling you exactly how much they understand. AI is not the line item. AI is the budget. The global economy is being re-denominated in a currency most people have not even heard of yet. Tokens. Whoever controls the supply of that currency is not playing in the new economy. They are the house. And the house does not lose.

Dustin

43,505 views • 4 months ago

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."
1:12

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Fireside Alpha

281,474 views • 13 days ago

Chase Hughes just revealed a wild way to edit traumatic memories using hypnosis on the Joe Rogan Experience. HUGHES: “I figured out a way to edit memory… I can take you back and change your perspective on trauma.” “You go back to that event when you got kicked in the nuts and everybody laughed at you in elementary school… and you can reprocess that memory in a very short amount of time as an adult with the perspective of an adult.” “Instead of modifying the memory like ‘oh that never happened’ — the memory stays. The perspective changes.” “And it’s exactly what psychedelics do. This massive perspective shift on memory… their whole life can be different after that day.” He says he’s done this hundreds of times and it creates real downstream effects.
5:35

Chase Hughes just revealed a wild way to edit traumatic memories using hypnosis on the Joe Rogan Experience. HUGHES: “I figured out a way to edit memory… I can take you back and change your perspective on trauma.” “You go back to that event when you got kicked in the nuts and everybody laughed at you in elementary school… and you can reprocess that memory in a very short amount of time as an adult with the perspective of an adult.” “Instead of modifying the memory like ‘oh that never happened’ — the memory stays. The perspective changes.” “And it’s exactly what psychedelics do. This massive perspective shift on memory… their whole life can be different after that day.” He says he’s done this hundreds of times and it creates real downstream effects.

Defiant Ghost

26,912 views • 1 month ago

"The magic of tokens that's not really viable for stocks is that you could keep minting on the other side... totally decoupled from the fees coming in." Eddy Lazzarin ☀️ on the buy-and-burn vs. mint-and-spend, and the lever token networks have that equity doesn't: "The idea of the buy and burn is that you take revenue in one side, you charge whatever you can get away with in terms of your price. You take that revenue and use it to burn the token... reducing its supply, which helps it appreciate." "The mirror of the buy and burn is the mint and spend. And this is where you are minting tokens out of nowhere to pay for whatever you need for growth or for network activities." "You can keep pulling in fees and test your ability to charge in the market while completely separately scaling your spending in order to facilitate the most growth." "In stocks, you can't pay an onboarding bonus to Uber drivers or whatever with your stock... To pay for growth, a conventional company has to do it in a very indirect way. They have to sell the equity to an investor in a private round to get cash to spend on growth. But this allows you to circumvent that completely."
1:43

"The magic of tokens that's not really viable for stocks is that you could keep minting on the other side... totally decoupled from the fees coming in." Eddy Lazzarin ☀️ on the buy-and-burn vs. mint-and-spend, and the lever token networks have that equity doesn't: "The idea of the buy and burn is that you take revenue in one side, you charge whatever you can get away with in terms of your price. You take that revenue and use it to burn the token... reducing its supply, which helps it appreciate." "The mirror of the buy and burn is the mint and spend. And this is where you are minting tokens out of nowhere to pay for whatever you need for growth or for network activities." "You can keep pulling in fees and test your ability to charge in the market while completely separately scaling your spending in order to facilitate the most growth." "In stocks, you can't pay an onboarding bonus to Uber drivers or whatever with your stock... To pay for growth, a conventional company has to do it in a very indirect way. They have to sell the equity to an investor in a private round to get cash to spend on growth. But this allows you to circumvent that completely."

a16z crypto

25,281 views • 1 month ago