Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

I joined AMD right after graduating from MIT. Two stints there and a decade building Cerebras later, I still lose sleep over how much memory and compute to put on a chip. I’ve spent my entire career in computer architecture, and I’ve never seen the problems change this quickly.... Models keep getting larger. The demands on the hardware keep growing. Decisions about where to put memory and how to connect it have become central to what people can build with AI. We chose SRAM for its bandwidth. Now we’re working on wafer-scale stacked DRAM to give larger models more capacity while preserving fast inference. That brings us back to some very physical problems: how to power it, cool it, and manufacture it reliably. This is the work I love. In part two of our AMA, I answered your questions about those decisions and where we’re going next. 0:30 What changes between CS4, CS5, and CS6? 2:19 HBM already stacks memory. What’s different here? 3:34 We chose SRAM for speed. Why add DRAM? 4:52 Drawing the stack is easy. Making it work isn’t. 7:23 The memory-versus-compute decision that keeps me up at nightshow more

Sean Lie

15,866 subscribers

30,739 görüntüleme • 15 gün önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 Yorum

Thayer profil fotoğrafı

Thayer15 gün önce

@cerebras Welcome to X and great interview!

DC3AI profil fotoğrafı

DC3AI14 gün önce

@cerebras What did you settle on?

Tai profil fotoğrafı

Tai14 gün önce

@cerebras On "manufacture it reliably": once DRAM sits on wafer-scale logic, a bad bond can't be binned away like a bad die. At volume, you patiently log failure modes and rates, decide per mode whether pre-stack test pays off, and keep revising the whole test flow as data comes in.

Benzer Videolar

When we started Cerebras, we knew we would have to solve difficult problems in yield, power, and cooling. What became clear was that making wafer-scale work would require rethinking almost the entire system. Today, we answered questions from the community about those engineering decisions, how we run larger models, and what it takes to program the hardware. 2:15 Why hasn't NVIDIA gone wafer-scale? 4:10 How we build a working chip out of an imperfect wafer 8:07 What happens when a model won't fit on one wafer? 11:34 Why we chose SRAM over HBM 14:26 How much of a barrier is NVIDIA's CUDA moat?

When we started Cerebras, we knew we would have to solve difficult problems in yield, power, and cooling. What became clear was that making wafer-scale work would require rethinking almost the entire system. Today, we answered questions from the community about those engineering decisions, how we run larger models, and what it takes to program the hardware. 2:15 Why hasn't NVIDIA gone wafer-scale? 4:10 How we build a working chip out of an imperfect wafer 8:07 What happens when a model won't fit on one wafer? 11:34 Why we chose SRAM over HBM 14:26 How much of a barrier is NVIDIA's CUDA moat?

Sean Lie

32,611 görüntüleme • 1 ay önce

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,935 görüntüleme • 4 ay önce

Gavin Baker says the piece missing from the compute debate is SRAM accelerators, and disaggregating inference across three chips lifts AI's ROI. "I do think something that is missing from all of this conversation about compute is what is going to happen when you put these SRAM-based accelerators that are not constrained by HBM DRAM and are often made on older nodes that are not competing with the latest greatest GPUs". "When you disaggregate inference, people talk about pre-fill and decode, but decode is two parts, attention and feed-forward network." "The ultimate holy grail is if you could do pre-fill on one chip that probably doesn't have HBM DRAM, do the attention on a super high-powered chip with HBM DRAM, and then do the feed-forward network on one of these SRAM chips." "You just can't beat SRAM in particular for that feed-forward network, and you just almost can't, no matter how much you try and get the ratio of compute to HBM DRAM to SRAM on the chip correct, the workloads are always changing, and there's different workloads." "Being able to disaggregate into these three parts, I think this is going to be really, really positive for the ROI on AI."

Gavin Baker says the piece missing from the compute debate is SRAM accelerators, and disaggregating inference across three chips lifts AI's ROI. "I do think something that is missing from all of this conversation about compute is what is going to happen when you put these SRAM-based accelerators that are not constrained by HBM DRAM and are often made on older nodes that are not competing with the latest greatest GPUs". "When you disaggregate inference, people talk about pre-fill and decode, but decode is two parts, attention and feed-forward network." "The ultimate holy grail is if you could do pre-fill on one chip that probably doesn't have HBM DRAM, do the attention on a super high-powered chip with HBM DRAM, and then do the feed-forward network on one of these SRAM chips." "You just can't beat SRAM in particular for that feed-forward network, and you just almost can't, no matter how much you try and get the ratio of compute to HBM DRAM to SRAM on the chip correct, the workloads are always changing, and there's different workloads." "Being able to disaggregate into these three parts, I think this is going to be really, really positive for the ROI on AI."

Fireside Alpha

15,501 görüntüleme • 2 ay önce

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."

Trade Whisperer

180,543 görüntüleme • 2 ay önce

Andrew Feldman, co-founder and CEO of Cerebras gives the best explanation of why Cerebras' wafer-scale architecture is 2,500X faster than a GPU during LLM inference. During inference, there are 2 stages: - pre-fill, where the model first processes the user's prompt, and - decode, where it generates the answer 1 token at a time in sequence. During that sequencial Decode phase, before each token is calculated the model weights have to be moved from memory into compute. On a GPU those weights are moved from HBM, while Cerebras keeps them in much faster SRAM spread across its very large wafer-scale processor, so the memory-to-compute movement that must happen for every token is about 2,500× faster ---- From The MAD Podcast with Matt Turck and Cerebras YouTube channel, (full video link in comment)

Andrew Feldman, co-founder and CEO of Cerebras gives the best explanation of why Cerebras' wafer-scale architecture is 2,500X faster than a GPU during LLM inference. During inference, there are 2 stages: - pre-fill, where the model first processes the user's prompt, and - decode, where it generates the answer 1 token at a time in sequence. During that sequencial Decode phase, before each token is calculated the model weights have to be moved from memory into compute. On a GPU those weights are moved from HBM, while Cerebras keeps them in much faster SRAM spread across its very large wafer-scale processor, so the memory-to-compute movement that must happen for every token is about 2,500× faster ---- From The MAD Podcast with Matt Turck and Cerebras YouTube channel, (full video link in comment)

Rohan Paul

39,999 görüntüleme • 1 ay önce

Etched is deploying two new technologies in chip design: low-voltage inference and cluster-scale memory. CEO Gavin Uberti says they'll make their chips much more power-efficient and way, way faster than today's leading GPUs. He breaks it down: "We looked at a lot of early research directions, and we realized the key things that models need are way more compute and way faster memory." "If you think about inference, there are two key parts: prefill and decode. For prefill, it's a compute-bound problem. You need to have more FLOPS, more operations per second on each of your chips." "On our GPU, the bottleneck's actually thermals. You can't really run a GPU at more than around 50% of what it could theoretically do, or it'll melt." "So we're using a new technology today called low-voltage inference to try to solve this problem. You bring the voltage of the chip down dramatically, which allows us to have way, way better efficiency in terms of how much power is drawn per unit of math, and thus fit way way more flops onto the chip..." "For decode, it's all about bandwidth. Not just bandwidth on a chip, but bandwidth across your cluster. That's why we have this technology we call cluster-scale memory. It reduces the amount of time it takes to communicate from one chip to another dramatically." "As a result we can go use all of our HBM, HBM bandwidth, SRAM, SRAM bandwidth, and our scale-up domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second, while still keeping your costs low."

Etched is deploying two new technologies in chip design: low-voltage inference and cluster-scale memory. CEO Gavin Uberti says they'll make their chips much more power-efficient and way, way faster than today's leading GPUs. He breaks it down: "We looked at a lot of early research directions, and we realized the key things that models need are way more compute and way faster memory." "If you think about inference, there are two key parts: prefill and decode. For prefill, it's a compute-bound problem. You need to have more FLOPS, more operations per second on each of your chips." "On our GPU, the bottleneck's actually thermals. You can't really run a GPU at more than around 50% of what it could theoretically do, or it'll melt." "So we're using a new technology today called low-voltage inference to try to solve this problem. You bring the voltage of the chip down dramatically, which allows us to have way, way better efficiency in terms of how much power is drawn per unit of math, and thus fit way way more flops onto the chip..." "For decode, it's all about bandwidth. Not just bandwidth on a chip, but bandwidth across your cluster. That's why we have this technology we call cluster-scale memory. It reduces the amount of time it takes to communicate from one chip to another dramatically." "As a result we can go use all of our HBM, HBM bandwidth, SRAM, SRAM bandwidth, and our scale-up domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second, while still keeping your costs low."

TBPN

20,404 görüntüleme • 3 ay önce

DeepMind CEO Demis Hassabis says the AI bottleneck is already moving from model ideas to physical supply chains. "How much of the restriction at this point is physical? I am talking memory shortage, GPU shortage, and electricity." Demis: "That is constraining a lot of the deployment for sure. We see so much more demand for our models - Gemini and the other models that we have - than we can possibly serve." Then the less obvious part: "It does constrain a little bit the research, because you need a lot of chips to be able to experiment on new ideas at big enough scale that you can actually see if they are going to work." And on memory: "The whole supply chain is kind of strained... It still in the end actually comes down to a few suppliers of a few of the key components. Anywhere where there is some kind of constraint on the capacity, there is a sort of choke point." Important framing: The next AI race is not just who has the best model architecture. It is who can secure memory, chips, power, suppliers, and enough experimental capacity to find the next breakthrough before everyone else.

DeepMind CEO Demis Hassabis says the AI bottleneck is already moving from model ideas to physical supply chains. "How much of the restriction at this point is physical? I am talking memory shortage, GPU shortage, and electricity." Demis: "That is constraining a lot of the deployment for sure. We see so much more demand for our models - Gemini and the other models that we have - than we can possibly serve." Then the less obvious part: "It does constrain a little bit the research, because you need a lot of chips to be able to experiment on new ideas at big enough scale that you can actually see if they are going to work." And on memory: "The whole supply chain is kind of strained... It still in the end actually comes down to a few suppliers of a few of the key components. Anywhere where there is some kind of constraint on the capacity, there is a sort of choke point." Important framing: The next AI race is not just who has the best model architecture. It is who can secure memory, chips, power, suppliers, and enough experimental capacity to find the next breakthrough before everyone else.

Karl Mehta

25,716 görüntüleme • 3 ay önce

#AbuDhabi 🇦🇪 | Lewis Hamilton on what’s next: “It’s not actually a straight forward process. We’re obviously testing next week and then we go back to the factory and I’ve gotta decide what my approach is, when it comes to sitting down with the stakeholders to make the decisions and how that approach is to create the change that’s needed.” “And then it terms of personal reflection… I mean, I literally every weekend have written down what I felt, what went wrong that weekend, the decisions I’ve taken so there’s a lot of learnings to do there. And I will, through my break, I will analyse those decisions and make markers on how I can make better decisions in the future. My surroundings in terms of personal personnel, team personnel, how you utilise people, whether people need to move into different positions to work better, all these different things need to be looked upon in my personal space so that we can optimise our teamwork.”

#AbuDhabi 🇦🇪 | Lewis Hamilton on what’s next: “It’s not actually a straight forward process. We’re obviously testing next week and then we go back to the factory and I’ve gotta decide what my approach is, when it comes to sitting down with the stakeholders to make the decisions and how that approach is to create the change that’s needed.” “And then it terms of personal reflection… I mean, I literally every weekend have written down what I felt, what went wrong that weekend, the decisions I’ve taken so there’s a lot of learnings to do there. And I will, through my break, I will analyse those decisions and make markers on how I can make better decisions in the future. My surroundings in terms of personal personnel, team personnel, how you utilise people, whether people need to move into different positions to work better, all these different things need to be looked upon in my personal space so that we can optimise our teamwork.”

deni

159,775 görüntüleme • 10 ay önce

Chamath: Two terms you need to pay attention to in AI are Prefill and Decode “There's two terms that I think you're going to hear a ton about over these next few years.” “The first term is prefill, and the next is decode.” “What prefill and decode are, are two very distinct ways of how models think, and how a model goes through the process of answering a question that you ask it.” “And so when you send a prompt to AI, what happens is that the model processes it. This is called the reading phase or prefill.” “It reads your entire prompt all at once. And then it does a bunch of math, calculates all these relationships between all the words, and it stores them in temporary memory.” “The problem is that this is really compute bound. So it requires massive brute force. And Nvidia GPUs crush here.” “And their architecture is designed for massive parallel processing, which makes them really amazing at digesting these long prompts.” “So the problem just gets bigger and bigger, Nvidia just completely dominates.” “But the next phase though, this critical phase, the decode phase, is the writing phase, right?” “So the model starts to generate a response, you ask it a question and its response, one token at a time.” “And then to pick the next token to pick the next word, it has to look back at everything it has said already so that it doesn't hallucinate.” “The problem is that this is incredibly memory bandwidth constrained.” “And in our architecture, a long time ago, we made these design decisions from day one.” “And so what we did was we took a very different architectural approach, we took a very conservative process technology. We weren't pushing the boundaries of physics.” “And we used a lot of what's called SRAM. So memory on the chip so that we could do this decode thing as well or better than everybody else.” “And so now when you put these two things together, I just think it's going to create a huge acceleration in the ability for this entire infrastructure layer to get much cheaper and much more valuable, which I suspect then it'll have a lot more developer pull, you'll get a lot more applications being built, billions and billions of more people using it.”

Chamath: Two terms you need to pay attention to in AI are Prefill and Decode “There's two terms that I think you're going to hear a ton about over these next few years.” “The first term is prefill, and the next is decode.” “What prefill and decode are, are two very distinct ways of how models think, and how a model goes through the process of answering a question that you ask it.” “And so when you send a prompt to AI, what happens is that the model processes it. This is called the reading phase or prefill.” “It reads your entire prompt all at once. And then it does a bunch of math, calculates all these relationships between all the words, and it stores them in temporary memory.” “The problem is that this is really compute bound. So it requires massive brute force. And Nvidia GPUs crush here.” “And their architecture is designed for massive parallel processing, which makes them really amazing at digesting these long prompts.” “So the problem just gets bigger and bigger, Nvidia just completely dominates.” “But the next phase though, this critical phase, the decode phase, is the writing phase, right?” “So the model starts to generate a response, you ask it a question and its response, one token at a time.” “And then to pick the next token to pick the next word, it has to look back at everything it has said already so that it doesn't hallucinate.” “The problem is that this is incredibly memory bandwidth constrained.” “And in our architecture, a long time ago, we made these design decisions from day one.” “And so what we did was we took a very different architectural approach, we took a very conservative process technology. We weren't pushing the boundaries of physics.” “And we used a lot of what's called SRAM. So memory on the chip so that we could do this decode thing as well or better than everybody else.” “And so now when you put these two things together, I just think it's going to create a huge acceleration in the ability for this entire infrastructure layer to get much cheaper and much more valuable, which I suspect then it'll have a lot more developer pull, you'll get a lot more applications being built, billions and billions of more people using it.”

The All-In Podcast

567,660 görüntüleme • 9 ay önce

Stephen Colbert: “As someone who is very qualified for the presidency… a very hopeful and dynamic presidential candidate for the 107 days you had to run. To hear you say that it’s broken. To hear you say that our systems aren’t strong is harrowing.” Kamala Harris: “Well, it’s also evident, isn’t it?... It doesn’t mean we give up. That’s not my point… I’m always going to be a part of the fight that’s never going to change… And I believe right now that it is important to do what I can do, from the positions that I’ve held and what I have seen about the world and our country, to get out there and remind everyone who needs reminding right now of their power.” “It is our government. It is our country, and it is important, I think, that in this moment where people have become so deflated and despondent and afraid, that those of us who have the ability, which I do right now, not being in an office where I’m campaigning for that office, to be out there and to talk with folks and remind them of their power and their importance in making a difference.” “I just want to put a fine point on this. You can never let anybody take your power from you. You can never let anybody take your power from you. And that’s what I think, for me, that’s what I like to remind folks of.”

Stephen Colbert: “As someone who is very qualified for the presidency… a very hopeful and dynamic presidential candidate for the 107 days you had to run. To hear you say that it’s broken. To hear you say that our systems aren’t strong is harrowing.” Kamala Harris: “Well, it’s also evident, isn’t it?... It doesn’t mean we give up. That’s not my point… I’m always going to be a part of the fight that’s never going to change… And I believe right now that it is important to do what I can do, from the positions that I’ve held and what I have seen about the world and our country, to get out there and remind everyone who needs reminding right now of their power.” “It is our government. It is our country, and it is important, I think, that in this moment where people have become so deflated and despondent and afraid, that those of us who have the ability, which I do right now, not being in an office where I’m campaigning for that office, to be out there and to talk with folks and remind them of their power and their importance in making a difference.” “I just want to put a fine point on this. You can never let anybody take your power from you. You can never let anybody take your power from you. And that’s what I think, for me, that’s what I like to remind folks of.”

RedWave Press

302,166 görüntüleme • 1 yıl önce

Discovered Materials' Akash Ramdas argues the next chip scaling frontier isn't smaller transistors, it's how many you can stack on top of each other: HBM is already 12 high DRAM going to 16 high "We're starting to do this thing where we're taking a logic chip and then putting memory on top of it. That's where materials can make a difference, because now you're actually thermally limited. The thing that's preventing your scaling is no longer how small is your transistor, but how many transistors can you put on top of each other?" "The high bandwidth memory you'd have seen is a stack of dynamic random access memory. It's a 12-high dynamic random access memory stack right now. One of the roadmaps is to make this even higher, 16-high, and this is how you deal with memory bottlenecks in inference or training." "The next stage of that is putting that HBM on top of a logic die. Both these technologies are going to be partially enabled by new materials."

Discovered Materials' Akash Ramdas argues the next chip scaling frontier isn't smaller transistors, it's how many you can stack on top of each other: HBM is already 12 high DRAM going to 16 high "We're starting to do this thing where we're taking a logic chip and then putting memory on top of it. That's where materials can make a difference, because now you're actually thermally limited. The thing that's preventing your scaling is no longer how small is your transistor, but how many transistors can you put on top of each other?" "The high bandwidth memory you'd have seen is a stack of dynamic random access memory. It's a 12-high dynamic random access memory stack right now. One of the roadmaps is to make this even higher, 16-high, and this is how you deal with memory bottlenecks in inference or training." "The next stage of that is putting that HBM on top of a logic die. Both these technologies are going to be partially enabled by new materials."

MTS

17,566 görüntüleme • 2 ay önce

Cerebras just IPO’d and the stock already ran up over 100% (Save this). For the entire 70 year history of the semiconductor industry, every company on earth has followed the same process. You take a dinner plate sized silicon wafer, put hundreds of tiny chips onto it, and dice it up like a pizza. Nvidia does it this way, AMD does it this way, Intel has done it this way for six decades and everyone who tried to break that convention failed. Until Cerebras asked the most annoyingly obvious question in the industry’s history, what if you just didn’t cut it? The result is the Wafer Scale Engine, a single chip 56 times larger than Nvidia’s H100 and it fundamentally changes the physics of how AI inference works. The reason this matters is not the size, it’s the bandwidth. Every time an AI model generates a single word, it has to reach into memory, pull weights, multiply them together, and produce a prediction and when you’re running millions of concurrent sessions at once, the bottleneck is not raw processing power but how fast data moves between memory and compute. Nvidia’s H100 moves data at roughly 3 terabytes per second, while Cerebras’ WSE-3 moves data at 21 petabytes per second, roughly 7,000 times faster because memory and compute live on the same enormous piece of silicon and data barely has to travel at all. That gap is exactly why OpenAI went from 150 tokens per second on traditional GPUs to 2,000 tokens per second on Cerebras hardware, and why AWS integrated Cerebras into Bedrock to deliver roughly 5x more inference capacity in the same physical footprint. The macro setup is making the trade even more urgent. South Korea DRAM export prices recently jumped 35%, flash memory surged 47%, and SSD pricing spiked nearly 140% and every single one of those increases hits Nvidia-based infrastructure directly, because the H100 requires 80GB of the most expensive, most contested memory in the AI supply chain. Cerebras’ WSE-3 uses zero external HBM memory, baking 44GB of SRAM directly into the wafer itself which means as memory pricing goes parabolic, every CFO evaluating AI infrastructure is suddenly looking much more seriously at the architecture that sidesteps that cost entirely. The demand is already showing up in the backlog. Cerebras ended 2025 with $24.6 billion in remaining performance obligations for a company doing just over $500 million in annual revenue, that is a number that implies years of contracted growth already sitting on the books. The IPO was 20x oversubscribed, the price range was raised twice before listing, and shares opened 89% above their listing price on a $5.55 billion raise that made it the largest semiconductor IPO in history. The risks are real and worth naming. 86% of 2025 revenue came from two entities with UAE ties, U.S. revenue actually fell 34% to $187 million, and the $20 billion OpenAI contract is conditional, if Cerebras misses delivery milestones, OpenAI can terminate and trigger repayment demands on a $1 billion loan facility. And yet the market is valuing Cerebras at roughly 91x trailing revenue, richer than Nvidia, AMD, and Arm combined. What investors are betting on is not that Cerebras beats Nvidia, it is that the inference supercycle is large enough to support an entirely different architecture optimized for a different workload, and that $24.6 billion in contracted backlog converts to diversified revenue before the market starts asking harder questions. CEO Andrew Feldman said this took a decade of late nights to get right, everyone who tried to copy it failed and given that the entire inference economy is now running through exactly the bottleneck Cerebras was built to eliminate, the market is starting to believe him.

Cerebras just IPO’d and the stock already ran up over 100% (Save this). For the entire 70 year history of the semiconductor industry, every company on earth has followed the same process. You take a dinner plate sized silicon wafer, put hundreds of tiny chips onto it, and dice it up like a pizza. Nvidia does it this way, AMD does it this way, Intel has done it this way for six decades and everyone who tried to break that convention failed. Until Cerebras asked the most annoyingly obvious question in the industry’s history, what if you just didn’t cut it? The result is the Wafer Scale Engine, a single chip 56 times larger than Nvidia’s H100 and it fundamentally changes the physics of how AI inference works. The reason this matters is not the size, it’s the bandwidth. Every time an AI model generates a single word, it has to reach into memory, pull weights, multiply them together, and produce a prediction and when you’re running millions of concurrent sessions at once, the bottleneck is not raw processing power but how fast data moves between memory and compute. Nvidia’s H100 moves data at roughly 3 terabytes per second, while Cerebras’ WSE-3 moves data at 21 petabytes per second, roughly 7,000 times faster because memory and compute live on the same enormous piece of silicon and data barely has to travel at all. That gap is exactly why OpenAI went from 150 tokens per second on traditional GPUs to 2,000 tokens per second on Cerebras hardware, and why AWS integrated Cerebras into Bedrock to deliver roughly 5x more inference capacity in the same physical footprint. The macro setup is making the trade even more urgent. South Korea DRAM export prices recently jumped 35%, flash memory surged 47%, and SSD pricing spiked nearly 140% and every single one of those increases hits Nvidia-based infrastructure directly, because the H100 requires 80GB of the most expensive, most contested memory in the AI supply chain. Cerebras’ WSE-3 uses zero external HBM memory, baking 44GB of SRAM directly into the wafer itself which means as memory pricing goes parabolic, every CFO evaluating AI infrastructure is suddenly looking much more seriously at the architecture that sidesteps that cost entirely. The demand is already showing up in the backlog. Cerebras ended 2025 with $24.6 billion in remaining performance obligations for a company doing just over $500 million in annual revenue, that is a number that implies years of contracted growth already sitting on the books. The IPO was 20x oversubscribed, the price range was raised twice before listing, and shares opened 89% above their listing price on a $5.55 billion raise that made it the largest semiconductor IPO in history. The risks are real and worth naming. 86% of 2025 revenue came from two entities with UAE ties, U.S. revenue actually fell 34% to $187 million, and the $20 billion OpenAI contract is conditional, if Cerebras misses delivery milestones, OpenAI can terminate and trigger repayment demands on a $1 billion loan facility. And yet the market is valuing Cerebras at roughly 91x trailing revenue, richer than Nvidia, AMD, and Arm combined. What investors are betting on is not that Cerebras beats Nvidia, it is that the inference supercycle is large enough to support an entirely different architecture optimized for a different workload, and that $24.6 billion in contracted backlog converts to diversified revenue before the market starts asking harder questions. CEO Andrew Feldman said this took a decade of late nights to get right, everyone who tried to copy it failed and given that the entire inference economy is now running through exactly the bottleneck Cerebras was built to eliminate, the market is starting to believe him.

Milk Road AI

30,441 görüntüleme • 5 ay önce

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 görüntüleme • 3 ay önce

Shaun Maguire on SpaceX IPO: "Starship is the railroad to space." "In 2019, it was very predictable that Starlink would work and produce a massive amount of revenue, and we called that right." "We're now in a very similar point to where Starlink was in 2019 with AI infrastructure and orbital compute." "I think people are really underestimating how far along the orbital compute product is right now." On the $22 trillion TAM: "What do you think is the TAM for intelligence? What's the TAM for all compute infrastructure 50 years from now?" "People are underappreciating how much infrastructure is going to be built in space and how much easier it is to access energy in space." "We're at the beginning of this AI era, and we're going to double or triple the total amount of intelligence that we have access to over the next decade — so yes, I absolutely believe in the TAM, and I would even argue it's an underestimate." How much is the "railroad to space" worth in the era of AI and orbital compute? Now imagine the railroad company also owns the AI and the orbital compute. Elon Musk Shaun Maguire

Shaun Maguire on SpaceX IPO: "Starship is the railroad to space." "In 2019, it was very predictable that Starlink would work and produce a massive amount of revenue, and we called that right." "We're now in a very similar point to where Starlink was in 2019 with AI infrastructure and orbital compute." "I think people are really underestimating how far along the orbital compute product is right now." On the $22 trillion TAM: "What do you think is the TAM for intelligence? What's the TAM for all compute infrastructure 50 years from now?" "People are underappreciating how much infrastructure is going to be built in space and how much easier it is to access energy in space." "We're at the beginning of this AI era, and we're going to double or triple the total amount of intelligence that we have access to over the next decade — so yes, I absolutely believe in the TAM, and I would even argue it's an underestimate." How much is the "railroad to space" worth in the era of AI and orbital compute? Now imagine the railroad company also owns the AI and the orbital compute. Elon Musk Shaun Maguire

KanekoaTheGreat

39,709 görüntüleme • 4 ay önce

Yesterday, I joined TBPN to talk about what it really means to give AI a memory. And, why that’s becoming the next big moat. We discussed: - Why today’s AI apps are stateless - How memory is quickly becoming the real moat for AI systems - How Mem0 is decoupling memory from models - Why users should be able to own and carry their AI memories across apps - And, how 200 cold emails changed my life🙂 Appreciate John Coogan, Jordi Hays & the TBPN team for having me and for such a thoughtful discussion around the future of AI infrastructure.

Yesterday, I joined TBPN to talk about what it really means to give AI a memory. And, why that’s becoming the next big moat. We discussed: - Why today’s AI apps are stateless - How memory is quickly becoming the real moat for AI systems - How Mem0 is decoupling memory from models - Why users should be able to own and carry their AI memories across apps - And, how 200 cold emails changed my life🙂 Appreciate John Coogan, Jordi Hays & the TBPN team for having me and for such a thoughtful discussion around the future of AI infrastructure.

Taranjeet

14,913 görüntüleme • 11 ay önce

Jensen Huang just made a statement that every investor in AI infrastructure needs to hear (Save this). He said that the AI buildout is accelerating, the second half of this year is going to be much larger than the first half, and next year is going to be very, very large. Micron is the best positioned to win from this because every Nvidia GPU requires High Bandwidth Memory stacked directly on the chip to feed it data fast enough to keep up. There is no AI compute without memory, and right now there is simply not enough memory to go around. Micron's entire HBM supply for 2026 is already completely sold out under multi-year agreements before the year even started. Micron's own management has acknowledged they can only satisfy 50 to 65 percent of demand from some of their most important customers. That is not a problem that gets fixed quickly, because new fabs take years to build. Micron's Idaho expansion does not come online until mid-2026, a second Idaho facility is not expected until 2028, and a new New York fab is looking at 2030. The demand Jensen just described is arriving right now, and the supply to meet it is years away. The financial results already reflect this dynamic. Micron's Q2 fiscal 2026 revenue came in at $23.86 billion, nearly triple what it was a year earlier beating consensus by roughly $3.8 billion. The HBM market alone is expected to grow from $35 billion today to $100 billion by 2028, and Micron has been consistently ahead of that forecast. Jensen just told the world the second half of this year and all of next year are going to be larger than anything that came before. Micron is the company that supplies the memory those GPUs need to run, and it cannot build supply fast enough to keep up with demand. Come join Milk Road Pro for our full deep dive on Micron, the HBM supply thesis and our AI trade thesis! Link below!

Jensen Huang just made a statement that every investor in AI infrastructure needs to hear (Save this). He said that the AI buildout is accelerating, the second half of this year is going to be much larger than the first half, and next year is going to be very, very large. Micron is the best positioned to win from this because every Nvidia GPU requires High Bandwidth Memory stacked directly on the chip to feed it data fast enough to keep up. There is no AI compute without memory, and right now there is simply not enough memory to go around. Micron's entire HBM supply for 2026 is already completely sold out under multi-year agreements before the year even started. Micron's own management has acknowledged they can only satisfy 50 to 65 percent of demand from some of their most important customers. That is not a problem that gets fixed quickly, because new fabs take years to build. Micron's Idaho expansion does not come online until mid-2026, a second Idaho facility is not expected until 2028, and a new New York fab is looking at 2030. The demand Jensen just described is arriving right now, and the supply to meet it is years away. The financial results already reflect this dynamic. Micron's Q2 fiscal 2026 revenue came in at $23.86 billion, nearly triple what it was a year earlier beating consensus by roughly $3.8 billion. The HBM market alone is expected to grow from $35 billion today to $100 billion by 2028, and Micron has been consistently ahead of that forecast. Jensen just told the world the second half of this year and all of next year are going to be larger than anything that came before. Micron is the company that supplies the memory those GPUs need to run, and it cannot build supply fast enough to keep up with demand. Come join Milk Road Pro for our full deep dive on Micron, the HBM supply thesis and our AI trade thesis! Link below!

Milk Road AI

77,554 görüntüleme • 4 ay önce

SK Hynix's Hoshik Kim admits high bandwidth memory "is not the answer" to the memory-wall problem, and SK is working on different solutions to solve the bottleneck. "No, actually, I kind of agree with Pat. So HBM is not the answer." [SK Hynix is admitting that HBM is a bad memory?] "Nothing is a bad memory. It's a good memory, but it's not the final answer. So actually, HBM will not solve the memory-wall problem. Memory-wall problem is an inherent AI problem, which you cannot avoid." "So we try our best to provide higher bandwidth with this memory, with HBM technology even though it has some limitation, as Pat already mentioned, with thermal, IOs, cost, a lot of stuff. "But that's the only thing we got right now to help ease this memory-wall problem, memory-bottleneck problem. But at the same time, we are working in different directions to solve this memory-wall problem with different solutions."

SK Hynix's Hoshik Kim admits high bandwidth memory "is not the answer" to the memory-wall problem, and SK is working on different solutions to solve the bottleneck. "No, actually, I kind of agree with Pat. So HBM is not the answer." [SK Hynix is admitting that HBM is a bad memory?] "Nothing is a bad memory. It's a good memory, but it's not the final answer. So actually, HBM will not solve the memory-wall problem. Memory-wall problem is an inherent AI problem, which you cannot avoid." "So we try our best to provide higher bandwidth with this memory, with HBM technology even though it has some limitation, as Pat already mentioned, with thermal, IOs, cost, a lot of stuff. "But that's the only thing we got right now to help ease this memory-wall problem, memory-bottleneck problem. But at the same time, we are working in different directions to solve this memory-wall problem with different solutions."

Fireside Alpha

140,571 görüntüleme • 2 ay önce

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."

Fireside Alpha

66,227 görüntüleme • 3 ay önce

you let your agent keep notes, because the alternative was explaining yourself every morning it stopped asking, and it started repeating things that had stopped being true every guide on this covers what to save and where to put it, private for one agent, shared for the team not one of them covers how a fact leaves the agent writes its own memory while it works, so a guess it made on a tired tuesday gets stored in the same shape as something you told it, and after that the two are indistinguishable i counted mine this morning: forty files, six of them about decisions we reversed months ago, all forty read into every session since you did not give it a memory you gave it a record with no way to be corrected

you let your agent keep notes, because the alternative was explaining yourself every morning it stopped asking, and it started repeating things that had stopped being true every guide on this covers what to save and where to put it, private for one agent, shared for the team not one of them covers how a fact leaves the agent writes its own memory while it works, so a guess it made on a tired tuesday gets stored in the same shape as something you told it, and after that the two are indistinguishable i counted mine this morning: forty files, six of them about decisions we reversed months ago, all forty read into every session since you did not give it a memory you gave it a record with no way to be corrected

Lagosaur

13,908 görüntüleme • 1 ay önce