Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Micron’s new DRAM-based Compute Express Links (CXLs) are high-speed devices that help interconnect processors, accelerators, memory, storage and IO while simultaneously increasing compute efficiency and scalability. Learn More → Mouser Electronics

Supplyframe

4,012 subscribers

139,526 views • 1 year ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

NXP’s new Cortex-M33 triple-radio chips offer robust wired & wireless connectivity. The FRDM-RW612 Dev Board lets you evaluate RW61x chips with BLE, 5GHz WiFi & 802.15.4 for diverse applications. Learn More → Mouser Electronics

NXP’s new Cortex-M33 triple-radio chips offer robust wired & wireless connectivity. The FRDM-RW612 Dev Board lets you evaluate RW61x chips with BLE, 5GHz WiFi & 802.15.4 for diverse applications. Learn More → Mouser Electronics

Supplyframe

115,242 views • 1 year ago

Gavin Baker says the piece missing from the compute debate is SRAM accelerators, and disaggregating inference across three chips lifts AI's ROI. "I do think something that is missing from all of this conversation about compute is what is going to happen when you put these SRAM-based accelerators that are not constrained by HBM DRAM and are often made on older nodes that are not competing with the latest greatest GPUs". "When you disaggregate inference, people talk about pre-fill and decode, but decode is two parts, attention and feed-forward network." "The ultimate holy grail is if you could do pre-fill on one chip that probably doesn't have HBM DRAM, do the attention on a super high-powered chip with HBM DRAM, and then do the feed-forward network on one of these SRAM chips." "You just can't beat SRAM in particular for that feed-forward network, and you just almost can't, no matter how much you try and get the ratio of compute to HBM DRAM to SRAM on the chip correct, the workloads are always changing, and there's different workloads." "Being able to disaggregate into these three parts, I think this is going to be really, really positive for the ROI on AI."

Gavin Baker says the piece missing from the compute debate is SRAM accelerators, and disaggregating inference across three chips lifts AI's ROI. "I do think something that is missing from all of this conversation about compute is what is going to happen when you put these SRAM-based accelerators that are not constrained by HBM DRAM and are often made on older nodes that are not competing with the latest greatest GPUs". "When you disaggregate inference, people talk about pre-fill and decode, but decode is two parts, attention and feed-forward network." "The ultimate holy grail is if you could do pre-fill on one chip that probably doesn't have HBM DRAM, do the attention on a super high-powered chip with HBM DRAM, and then do the feed-forward network on one of these SRAM chips." "You just can't beat SRAM in particular for that feed-forward network, and you just almost can't, no matter how much you try and get the ratio of compute to HBM DRAM to SRAM on the chip correct, the workloads are always changing, and there's different workloads." "Being able to disaggregate into these three parts, I think this is going to be really, really positive for the ROI on AI."

Fireside Alpha

12,504 views • 18 hours ago

Edge computing and AI go hand-in-hand. Microchip's AI development tools are powering AI applications in industrial automation, advanced medical devices, and wearable technology. Learn More → MicrochipTechnologyInc.

Edge computing and AI go hand-in-hand. Microchip's AI development tools are powering AI applications in industrial automation, advanced medical devices, and wearable technology. Learn More → MicrochipTechnologyInc.

Supplyframe

402,457 views • 1 year ago

Electricity demand was essentially flat for decades. Now it’s exploding. Heron Power CEO Drew Baglino: "Energy efficiency more or less negated the growth in energy services." "The net electricity delivered to accomplish those energy services stayed basically flat." "Even with all the AI things that are going on… And that has allowed more and more compute capability for the same percentage of GDP electricity production." "It's changing now that we can find more applications for AI than ever before." "We are seeing electricity growth higher than it's ever been before." "It's because we are seeing broad based demand for more electricity — compute, transportation, industrial electrification." Drew Baglino Heron Power Electronics

Electricity demand was essentially flat for decades. Now it’s exploding. Heron Power CEO Drew Baglino: "Energy efficiency more or less negated the growth in energy services." "The net electricity delivered to accomplish those energy services stayed basically flat." "Even with all the AI things that are going on… And that has allowed more and more compute capability for the same percentage of GDP electricity production." "It's changing now that we can find more applications for AI than ever before." "We are seeing electricity growth higher than it's ever been before." "It's because we are seeing broad based demand for more electricity — compute, transportation, industrial electrification." Drew Baglino Heron Power Electronics

a16z

62,254 views • 4 months ago

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Fireside Alpha

281,474 views • 9 days ago

The world is rapidly changing to adapt to climate change. Sustainable Futures highlights companies and individuals leading the way in developing green technologies and ushering in more sustainable practices. Learn More → DigiKey Analog Devices, Inc.

The world is rapidly changing to adapt to climate change. Sustainable Futures highlights companies and individuals leading the way in developing green technologies and ushering in more sustainable practices. Learn More → DigiKey Analog Devices, Inc.

Supplyframe

1,530,942 views • 1 year ago

If $MU says NAND pricing is still strong, enterprise SSD demand is picking up and AI inference is creating more storage demand then $SNDK should benefit because SanDisk is one of the more direct ways to own that part of the cycle. Hyperscalers are still figuring out the right mix of DRAM, HBM, TLC NAND, QLC NAND and other storage architectures which can keep supply tighter for longer as memory companies are going to be hesitant to add too much capacity before demand becomes clearer. I'm going to be listening closely for Micron’s QLC commentary because QLC NAND for read-heavy inference workloads is the big driver and if Micron calls that out then it would show that the storage side of the AI memory story is still earlier than DRAM.

If $MU says NAND pricing is still strong, enterprise SSD demand is picking up and AI inference is creating more storage demand then $SNDK should benefit because SanDisk is one of the more direct ways to own that part of the cycle. Hyperscalers are still figuring out the right mix of DRAM, HBM, TLC NAND, QLC NAND and other storage architectures which can keep supply tighter for longer as memory companies are going to be hesitant to add too much capacity before demand becomes clearer. I'm going to be listening closely for Micron’s QLC commentary because QLC NAND for read-heavy inference workloads is the big driver and if Micron calls that out then it would show that the storage side of the AI memory story is still earlier than DRAM.

Shay Boloor

108,953 views • 1 month ago

Ripple bugs are aquatic insects that can dart across flowing streams with mesmerizing speed and agility. Using high-speed cameras, new research studies the fan structures that these bugs use to dance on water. Learn more:

Ripple bugs are aquatic insects that can dart across flowing streams with mesmerizing speed and agility. Using high-speed cameras, new research studies the fan structures that these bugs use to dance on water. Learn more:

Science Magazine

42,666 views • 9 months ago

Building AI infrastructure at yotta-scale requires more than raw performance; it demands an open, modular rack design that can evolve across product generations, combining leadership compute engines with high-speed networking to connect thousands of accelerators into a single, unified system. Take a look at the Helios. Our rack-scale AI platform powered by MI455X GPUs and EPYC “Venice” CPUs and designed for trillion-parameter training.

Building AI infrastructure at yotta-scale requires more than raw performance; it demands an open, modular rack design that can evolve across product generations, combining leadership compute engines with high-speed networking to connect thousands of accelerators into a single, unified system. Take a look at the Helios. Our rack-scale AI platform powered by MI455X GPUs and EPYC “Venice” CPUs and designed for trillion-parameter training.

AMD

20,832 views • 7 months ago

SITUATION EXPLAINED: Dwarkesh argues compute could get 10X more expensive. • Anthropic's revenue has been 10X-ing year over year while lab compute only 3X's • Three ways that gap can close: margins rise, compute gets more expensive, or labs shift compute to inference • All three are already happening, Anthropic went from 40% margins in 2025 to possibly 80%+ this year • But labs don't want the third one, heavy inference spend signals AI progress has stalled and you're now a cloud provider • Spot compute prices are up 40%+ since February, and labs pay well above spot for security and scale • Google is reportedly paying SpaceXAI $900 million a month for 110K GPUs, roughly 2X spot • Key claim: if a human-level software engineer ran on an H100, that H100 should rent for $250K a year, 15X today's price • The 3X annual compute growth is 1.4X Moore's Law, 1.2X new fabs, and 1.8X from AI taking wafer allocation from other devices • The fab piece is bottlenecked by EUV tool supply through 2030, and the wafer piece hits a wall by end of 2027 • If compute stays scarce, new labs need far more capital just to reach the same starting line • Compute gets cheap again only once robots can turn sand and copper into computers, which is gated on robotics, not RSI sof 𓋹: "This is what I'm most concerned about, a lack of innovation, not within companies, but a lack of new companies that can actually do something substantially new, because of the scarcity of compute." Theo Jaffee: "The biggest companies in the world by revenue are Amazon and Walmart, at 743 billion and 725 billion. If Anthropic makes 100 billion by the end of this year, that puts them at Target. To go from Target to bigger than Walmart in a year would be very impressive."

SITUATION EXPLAINED: Dwarkesh argues compute could get 10X more expensive. • Anthropic's revenue has been 10X-ing year over year while lab compute only 3X's • Three ways that gap can close: margins rise, compute gets more expensive, or labs shift compute to inference • All three are already happening, Anthropic went from 40% margins in 2025 to possibly 80%+ this year • But labs don't want the third one, heavy inference spend signals AI progress has stalled and you're now a cloud provider • Spot compute prices are up 40%+ since February, and labs pay well above spot for security and scale • Google is reportedly paying SpaceXAI $900 million a month for 110K GPUs, roughly 2X spot • Key claim: if a human-level software engineer ran on an H100, that H100 should rent for $250K a year, 15X today's price • The 3X annual compute growth is 1.4X Moore's Law, 1.2X new fabs, and 1.8X from AI taking wafer allocation from other devices • The fab piece is bottlenecked by EUV tool supply through 2030, and the wafer piece hits a wall by end of 2027 • If compute stays scarce, new labs need far more capital just to reach the same starting line • Compute gets cheap again only once robots can turn sand and copper into computers, which is gated on robotics, not RSI sof 𓋹: "This is what I'm most concerned about, a lack of innovation, not within companies, but a lack of new companies that can actually do something substantially new, because of the scarcity of compute." Theo Jaffee: "The biggest companies in the world by revenue are Amazon and Walmart, at 743 billion and 725 billion. If Anthropic makes 100 billion by the end of this year, that puts them at Target. To go from Target to bigger than Walmart in a year would be very impressive."

MTS

10,617 views • 4 days ago

The CEO of the world's largest asset manager just said something that should reframe how every investor thinks about the AI trade. Larry Fink, managing $11.5 trillion at BlackRock, stood at the Milken Institute Global Conference and said four words that matter, "We just don't have enough compute." "The United States is short power. We're short compute. We're short chips. And there's going to be shortages in all three and memory, four things. I actually believe a new asset class will be buying futures of compute." Think about what that means. Fink is predicting that compute becomes a tradable commodity like oil, like grain, like natural gas where investors buy forward contracts on future capacity because the shortage is so structural and so predictable that a derivatives market will emerge to price it. That is not a minor observation from a finance executive but rather the chairman of the most powerful capital allocator on the planet telling you that compute scarcity is a multi-year, investable megatrend. The data backs him up completely. Data centers will consume 70% of all memory chips produced globally in 2026. Advanced HBM production from Samsung, SK Hynix, and Micron is sold out through 2026 and into 2027 and a single AI server consumes 10-20x more memory than a conventional workload server. DRAM supply growth is running at just 16% annually while AI infrastructure demand is growing at 80%+. The chip crunch, the power crunch, and the compute crunch are not temporary dislocations, they are structural, and they will get worse before they get better. Fink also said something the bears keep getting wrong: "There is not an AI bubble. There is the opposite. We have supply shortages. Demand is growing much faster than anyone has ever anticipated." This is why the Milk Road Pro portfolio is built the way it is, long the companies producing and supplying the constrained resources: chips, memory, compute infrastructure, and power. Check out Milk Road Pro, link below to access our full thesis and plays.

The CEO of the world's largest asset manager just said something that should reframe how every investor thinks about the AI trade. Larry Fink, managing $11.5 trillion at BlackRock, stood at the Milken Institute Global Conference and said four words that matter, "We just don't have enough compute." "The United States is short power. We're short compute. We're short chips. And there's going to be shortages in all three and memory, four things. I actually believe a new asset class will be buying futures of compute." Think about what that means. Fink is predicting that compute becomes a tradable commodity like oil, like grain, like natural gas where investors buy forward contracts on future capacity because the shortage is so structural and so predictable that a derivatives market will emerge to price it. That is not a minor observation from a finance executive but rather the chairman of the most powerful capital allocator on the planet telling you that compute scarcity is a multi-year, investable megatrend. The data backs him up completely. Data centers will consume 70% of all memory chips produced globally in 2026. Advanced HBM production from Samsung, SK Hynix, and Micron is sold out through 2026 and into 2027 and a single AI server consumes 10-20x more memory than a conventional workload server. DRAM supply growth is running at just 16% annually while AI infrastructure demand is growing at 80%+. The chip crunch, the power crunch, and the compute crunch are not temporary dislocations, they are structural, and they will get worse before they get better. Fink also said something the bears keep getting wrong: "There is not an AI bubble. There is the opposite. We have supply shortages. Demand is growing much faster than anyone has ever anticipated." This is why the Milk Road Pro portfolio is built the way it is, long the companies producing and supplying the constrained resources: chips, memory, compute infrastructure, and power. Check out Milk Road Pro, link below to access our full thesis and plays.

Milk Road AI

419,283 views • 2 months ago

Autonomys is a purpose-built blockchain for AI infrastructure—a vertically integrated stack for decentralized storage and compute. ⛓️ Consensus: Built with the Substrate framework (sovereign L1, not part of Polkadot), powered by Proof-of-Archival-Storage. 🟦 Domains Architecture: Our modular execution layer for compute. The first Domain, Auto EVM, launched in July—permissionless smart contracts are live today. 🗺️ Roadmap: Permissionless instantiation of Domains, allowing developers to launch custom environments tailored to specific AI functions. Designed before AI became fashionable, Autonomys delivers the foundation that decentralized AI truly needs. 👉 Learn more:

Autonomys is a purpose-built blockchain for AI infrastructure—a vertically integrated stack for decentralized storage and compute. ⛓️ Consensus: Built with the Substrate framework (sovereign L1, not part of Polkadot), powered by Proof-of-Archival-Storage. 🟦 Domains Architecture: Our modular execution layer for compute. The first Domain, Auto EVM, launched in July—permissionless smart contracts are live today. 🗺️ Roadmap: Permissionless instantiation of Domains, allowing developers to launch custom environments tailored to specific AI functions. Designed before AI became fashionable, Autonomys delivers the foundation that decentralized AI truly needs. 👉 Learn more:

Autonomys | AI3.0

152,990 views • 9 months ago

Cookie cutter doesn't cut it anymore. Vik Malyala on how Supermicro became the go-to tier one OEM for startups building unconventional server architectures, from PCIe topologies to ARM-based compute. "Supermicro is the only vendor, sort of tier one OEM, who's able to both have the supply chain deliver anywhere in the world, whether it be US or Asia or Europe or Latam even, but also with a very unique server architecture. It's not just cookie cutter." "When you talk about the PCIe accelerators, these are the ones that are using these accelerators as a part of the system. And I have seen customers looking to have different PCIe topologies, whether it's a single root complex or multi root complex." "Initially we started with both Intel and AMD as a processor architecture to support these accelerators, but now we have added the new ARM based platforms as well. So now people can have an option on the compute platform as well as what accelerators they can support."

Cookie cutter doesn't cut it anymore. Vik Malyala on how Supermicro became the go-to tier one OEM for startups building unconventional server architectures, from PCIe topologies to ARM-based compute. "Supermicro is the only vendor, sort of tier one OEM, who's able to both have the supply chain deliver anywhere in the world, whether it be US or Asia or Europe or Latam even, but also with a very unique server architecture. It's not just cookie cutter." "When you talk about the PCIe accelerators, these are the ones that are using these accelerators as a part of the system. And I have seen customers looking to have different PCIe topologies, whether it's a single root complex or multi root complex." "Initially we started with both Intel and AMD as a processor architecture to support these accelerators, but now we have added the new ARM based platforms as well. So now people can have an option on the compute platform as well as what accelerators they can support."

SemiAnalysis

43,183 views • 13 days ago

🚀 Elon Musk has confirmed STARMIND as the name of SpaceX's future AI satellite network. FutureAzA says this is just one piece of a much bigger strategy. SpaceX is rapidly building an AI compute business, with reported customers including Google, Anthropic, and Reflection AI. The Reflection agreement alone is reportedly worth up to $6.3 billion through 2029. At the same time, Tesla continues expanding its energy business with a 25 GWh storage agreement through NatPower and a new partnership with Sunrun and Renew Home to help power AI infrastructure. AI, energy, compute, and space are starting to come together into one ecosystem. The more you zoom out, the more connected Tesla and SpaceX begin to look. $TSLA $SPCX

🚀 Elon Musk has confirmed STARMIND as the name of SpaceX's future AI satellite network. FutureAzA says this is just one piece of a much bigger strategy. SpaceX is rapidly building an AI compute business, with reported customers including Google, Anthropic, and Reflection AI. The Reflection agreement alone is reportedly worth up to $6.3 billion through 2029. At the same time, Tesla continues expanding its energy business with a 25 GWh storage agreement through NatPower and a new partnership with Sunrun and Renew Home to help power AI infrastructure. AI, energy, compute, and space are starting to come together into one ecosystem. The more you zoom out, the more connected Tesla and SpaceX begin to look. $TSLA $SPCX

Herbert Ong

15,014 views • 1 month ago

How to build a new AI paradigm, with Danyal A deeply ambitious new AI infrastructure company, based in London, launches today. Callosum has just raised $10.25m, led by Plural, to unlock a world of heterogeneous compute that optimises between performance, cost, speed + sovereignty If they're right, the implications go well beyond efficiency. If Callosum wins, they will build a global company by redistributing leverage globally, breaking our dependency on today’s dominant AI paradigm & rebuilding optionality across the world 00:00 The Brain as an Analogy for AI 04:38 Why One Big Model Fails 06:57 The Heterogeneous Compute Spectrum, from Datacentres to Edge Computing 11:54 Orchestrating Chips and Models 16:58 Supply, Demand and the Hardware Lottery 29:36 Value Beyond Chips 30:43 Bringing Chips to Market 35:50 Dynamic Compute Future 39:22 Funding and Roadmap 43:39 Riskiest Assumptions and Scaling the Team

How to build a new AI paradigm, with Danyal A deeply ambitious new AI infrastructure company, based in London, launches today. Callosum has just raised $10.25m, led by Plural, to unlock a world of heterogeneous compute that optimises between performance, cost, speed + sovereignty If they're right, the implications go well beyond efficiency. If Callosum wins, they will build a global company by redistributing leverage globally, breaking our dependency on today’s dominant AI paradigm & rebuilding optionality across the world 00:00 The Brain as an Analogy for AI 04:38 Why One Big Model Fails 06:57 The Heterogeneous Compute Spectrum, from Datacentres to Edge Computing 11:54 Orchestrating Chips and Models 16:58 Supply, Demand and the Hardware Lottery 29:36 Value Beyond Chips 30:43 Bringing Chips to Market 35:50 Dynamic Compute Future 39:22 Funding and Roadmap 43:39 Riskiest Assumptions and Scaling the Team

Andrew Bennett

11,687 views • 5 months ago

$MU $00060 $00530 $TSM Evolution of the AI Memory Hierarchy: HBM, DRAM, & SRAM Overview and Market Analysis The video outlines the strategic shift of modern computing toward a memory-centric architecture driven by the demands of generative AI. It explains that the industry is moving away from a traditional commodity model to a tightly coupled hierarchy of specialized technologies, where High Bandwidth Memory (HBM) and advanced packaging act as primary bottlenecks for performance. Leading manufacturers like SK hynix, Samsung, and Micron are navigating complex transitions to HBM4 and CXL-based expansion to overcome data movement constraints. Beyond the memory producers, the ecosystem relies on integration enablers like TSMC for packaging and specialized equipment providers for high-precision manufacturing. Ultimately, the analysis suggests that memory intensity and power efficiency have become the decisive factors in the success of gigawatt-scale data centers.

$MU $00060 $00530 $TSM Evolution of the AI Memory Hierarchy: HBM, DRAM, & SRAM Overview and Market Analysis The video outlines the strategic shift of modern computing toward a memory-centric architecture driven by the demands of generative AI. It explains that the industry is moving away from a traditional commodity model to a tightly coupled hierarchy of specialized technologies, where High Bandwidth Memory (HBM) and advanced packaging act as primary bottlenecks for performance. Leading manufacturers like SK hynix, Samsung, and Micron are navigating complex transitions to HBM4 and CXL-based expansion to overcome data movement constraints. Beyond the memory producers, the ecosystem relies on integration enablers like TSMC for packaging and specialized equipment providers for high-precision manufacturing. Ultimately, the analysis suggests that memory intensity and power efficiency have become the decisive factors in the success of gigawatt-scale data centers.

TheValueist

23,901 views • 7 months ago

1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering Contributions: • We delve into the temporal redundancy of 4D Gaussian Splatting and explain the main reason for the storage pressure and suboptimal rendering speed. • We introduce 4DGS-1K, a compact and memory-efficient framework to address these issues. It consists of two key components: a spatial-temporal variation score-based pruning strategy and a temporal filter. • Extensive experiments demonstrate that 4DGS-1K not only achieves a substantial storage reduction of approximately 41× but also accelerates rasterization to 1000+ FPS while maintaining high-quality reconstruction.

1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering Contributions: • We delve into the temporal redundancy of 4D Gaussian Splatting and explain the main reason for the storage pressure and suboptimal rendering speed. • We introduce 4DGS-1K, a compact and memory-efficient framework to address these issues. It consists of two key components: a spatial-temporal variation score-based pruning strategy and a temporal filter. • Extensive experiments demonstrate that 4DGS-1K not only achieves a substantial storage reduction of approximately 41× but also accelerates rasterization to 1000+ FPS while maintaining high-quality reconstruction.

MrNeRF

12,366 views • 1 year ago

ELON: AI CLUSTERS WILL BECOME A NORMAL THING EVERY COUNTRY HAS Compute is becoming a new currency. While building a "frontier model" today is a high-stakes gauntlet that requires a level of technical skill only a few companies possess, he predicts that AI compute clusters are about to become as essential to a nation as a power grid or a military. By 2026, "AI Sovereignty" will be the only thing keeping nations from becoming digital colonies of Silicon Valley or Beijing: “Probably all countries will have their own AI clusters over time. It is currently very difficult to actually build an AI cluster and have it run... because if you are training a frontier model, then you need a massive amount of compute and a level of technical skill that only a few companies possess. But over time, I think every country will have AI compute clusters. It is just going to be a normal thing that every country has.” Source: vitrupo Elon Musk

ELON: AI CLUSTERS WILL BECOME A NORMAL THING EVERY COUNTRY HAS Compute is becoming a new currency. While building a "frontier model" today is a high-stakes gauntlet that requires a level of technical skill only a few companies possess, he predicts that AI compute clusters are about to become as essential to a nation as a power grid or a military. By 2026, "AI Sovereignty" will be the only thing keeping nations from becoming digital colonies of Silicon Valley or Beijing: “Probably all countries will have their own AI clusters over time. It is currently very difficult to actually build an AI cluster and have it run... because if you are training a frontier model, then you need a massive amount of compute and a level of technical skill that only a few companies possess. But over time, I think every country will have AI compute clusters. It is just going to be a normal thing that every country has.” Source: vitrupo Elon Musk

Mario Nawfal

20,502 views • 7 months ago

Discover the tech, infrastructure, and innovation behind robotic manufacturing—because the future of automation is more than just the robot. Learn More → DigiKey Eaton SICK, Inc.

Discover the tech, infrastructure, and innovation behind robotic manufacturing—because the future of automation is more than just the robot. Learn More → DigiKey Eaton SICK, Inc.

Supplyframe

399,671 views • 1 year ago

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,678 views • 1 month ago