Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

When we started Cerebras, we knew we would have to solve difficult problems in yield, power, and cooling. What became clear was that making wafer-scale work would require rethinking almost the entire system. Today, we answered questions from the community about those engineering decisions, how we run larger models,... show more

Sean Lie

15,863 subscribers

32,611 views • 1 month ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

12 Comments

Papa Johns1 month ago

@cerebras @MilksandMatcha Thanks, Sean, for the insightful sharing. 👏

Abhishek Tiwari1 month ago

@cerebras @MilksandMatcha I like how this is very accessible for non super technical folks.

Sean Lie1 month ago

@cerebras @MilksandMatcha thank you

Sanjeev1 month ago

@cerebras @MilksandMatcha Very informative!

Engin1 month ago

@cerebras @MilksandMatcha Because it is really cool!

Mustafa1 month ago

@cerebras @MilksandMatcha The telescope became powerful not merely because its glass improved, but because the entire observatory was designed around precision.

Zeek | Insights 📊1 month ago

@BitcoinPierre @cerebras @MilksandMatcha Let me dive in

Michael Farber1 month ago

@cerebras @MilksandMatcha Super helpful.

Elias Mufarech1 month ago

@cerebras @MilksandMatcha good example of how every company is now a media company... and how easy it is to do it!

am.will1 month ago

@cerebras @MilksandMatcha great video!

Priyank Mishra1 month ago

@cerebras @MilksandMatcha the SRAM over HBM choice is the interesting one. if wafer-scale actually scales, some of the HBM demand story has a substitute nobody's pricing in. tiny share today, but it's a real one.

Anton Trunov1 month ago

@cerebras @MilksandMatcha Amazing overview! One of my friends got really excited when I joined Cerebras and has been nudging me since then to ask a cooling-related question: have you considered using 3M Novec or something better in the future?

Related Videos

I joined AMD right after graduating from MIT. Two stints there and a decade building Cerebras later, I still lose sleep over how much memory and compute to put on a chip. I’ve spent my entire career in computer architecture, and I’ve never seen the problems change this quickly. Models keep getting larger. The demands on the hardware keep growing. Decisions about where to put memory and how to connect it have become central to what people can build with AI. We chose SRAM for its bandwidth. Now we’re working on wafer-scale stacked DRAM to give larger models more capacity while preserving fast inference. That brings us back to some very physical problems: how to power it, cool it, and manufacture it reliably. This is the work I love. In part two of our AMA, I answered your questions about those decisions and where we’re going next. 0:30 What changes between CS4, CS5, and CS6? 2:19 HBM already stacks memory. What’s different here? 3:34 We chose SRAM for speed. Why add DRAM? 4:52 Drawing the stack is easy. Making it work isn’t. 7:23 The memory-versus-compute decision that keeps me up at night

I joined AMD right after graduating from MIT. Two stints there and a decade building Cerebras later, I still lose sleep over how much memory and compute to put on a chip. I’ve spent my entire career in computer architecture, and I’ve never seen the problems change this quickly. Models keep getting larger. The demands on the hardware keep growing. Decisions about where to put memory and how to connect it have become central to what people can build with AI. We chose SRAM for its bandwidth. Now we’re working on wafer-scale stacked DRAM to give larger models more capacity while preserving fast inference. That brings us back to some very physical problems: how to power it, cool it, and manufacture it reliably. This is the work I love. In part two of our AMA, I answered your questions about those decisions and where we’re going next. 0:30 What changes between CS4, CS5, and CS6? 2:19 HBM already stacks memory. What’s different here? 3:34 We chose SRAM for speed. Why add DRAM? 4:52 Drawing the stack is easy. Making it work isn’t. 7:23 The memory-versus-compute decision that keeps me up at night

Sean Lie

30,739 views • 15 days ago

we sent 2 PMs to Microsoft Build with one mission: whoever gets the most photos with the Wafer-Scale Engine in 10 minuntes, wins. for anyone who hasn't met it in person — the WSE is the largest chip ever built, and it powers the fastest AI. It tends to draw a crowd!

we sent 2 PMs to Microsoft Build with one mission: whoever gets the most photos with the Wafer-Scale Engine in 10 minuntes, wins. for anyone who hasn't met it in person — the WSE is the largest chip ever built, and it powers the fastest AI. It tends to draw a crowd!

Cerebras

13,039 views • 4 months ago

Zohran Mamdani on his vision for the future of the Democratic Party: “When we think about the New Deal, when we think about what we could deliver and the scale of what we deliver for working people, that’s what we have to bring back to our politics because for too long it’s felt as if politics is a hobby for those who can afford to have it not something that could deliver a better life for those who are being priced out of one today”

Zohran Mamdani on his vision for the future of the Democratic Party: “When we think about the New Deal, when we think about what we could deliver and the scale of what we deliver for working people, that’s what we have to bring back to our politics because for too long it’s felt as if politics is a hobby for those who can afford to have it not something that could deliver a better life for those who are being priced out of one today”

Really American 🇺🇸

19,386 views • 2 months ago

Tedros: "[The WHO] is working... to understand how misinformation and disinformation spreads... and what we can do to counter this problem." "But we must also make sure that when when we seek the trust of others, we are ourselves trustworthy." On a scale of 0-100, how "trustworthy" do you find Tedros and the WHO? 🤔

Tedros: "[The WHO] is working... to understand how misinformation and disinformation spreads... and what we can do to counter this problem." "But we must also make sure that when when we seek the trust of others, we are ourselves trustworthy." On a scale of 0-100, how "trustworthy" do you find Tedros and the WHO? 🤔

Wide Awake Media

86,445 views • 1 year ago

Rep. Roy: “And it goes beyond Fauci…Why aren't we hauling Deborah Birx in front of a committee? Why don't we haul a whole host of the bureaucrats at the NIH and the CDC who were all very well advised of what was going on. How about those federal employees that were trying to get the grants for Fauci? Why don't we look to see what they knew and when they knew it?”

Rep. Roy: “And it goes beyond Fauci…Why aren't we hauling Deborah Birx in front of a committee? Why don't we haul a whole host of the bureaucrats at the NIH and the CDC who were all very well advised of what was going on. How about those federal employees that were trying to get the grants for Fauci? Why don't we look to see what they knew and when they knew it?”

Rep. Chip Roy Press Office

11,680 views • 2 months ago

Andrew Feldman, co-founder and CEO of Cerebras gives the best explanation of why Cerebras' wafer-scale architecture is 2,500X faster than a GPU during LLM inference. During inference, there are 2 stages: - pre-fill, where the model first processes the user's prompt, and - decode, where it generates the answer 1 token at a time in sequence. During that sequencial Decode phase, before each token is calculated the model weights have to be moved from memory into compute. On a GPU those weights are moved from HBM, while Cerebras keeps them in much faster SRAM spread across its very large wafer-scale processor, so the memory-to-compute movement that must happen for every token is about 2,500× faster ---- From The MAD Podcast with Matt Turck and Cerebras YouTube channel, (full video link in comment)

Andrew Feldman, co-founder and CEO of Cerebras gives the best explanation of why Cerebras' wafer-scale architecture is 2,500X faster than a GPU during LLM inference. During inference, there are 2 stages: - pre-fill, where the model first processes the user's prompt, and - decode, where it generates the answer 1 token at a time in sequence. During that sequencial Decode phase, before each token is calculated the model weights have to be moved from memory into compute. On a GPU those weights are moved from HBM, while Cerebras keeps them in much faster SRAM spread across its very large wafer-scale processor, so the memory-to-compute movement that must happen for every token is about 2,500× faster ---- From The MAD Podcast with Matt Turck and Cerebras YouTube channel, (full video link in comment)

Rohan Paul

39,999 views • 1 month ago

Right now, when you send a query to an LLM, it gets decrypted on the server. The LLM sees your data in plain text. Prof. Ajay Joshi (BU, CipherSonic AI ) on fully homomorphic encryption, which may be key for the future of AI privacy: how we can compute on data without ever decrypting it. The catch: it's a brutally memory-bound workload. Exactly the bottleneck wafer-scale was built to solve.

Right now, when you send a query to an LLM, it gets decrypted on the server. The LLM sees your data in plain text. Prof. Ajay Joshi (BU, CipherSonic AI ) on fully homomorphic encryption, which may be key for the future of AI privacy: how we can compute on data without ever decrypting it. The catch: it's a brutally memory-bound workload. Exactly the bottleneck wafer-scale was built to solve.

Cerebras

293,785 views • 3 months ago

"What is P vs NP and how does it relate to human knowledge?" Avi Wigderson (Turing Award & Abel Prize winner): "In a very basic way, all the problems we want to solve are really NP problems. A subset of it is the problems we can solve. So the question of P versus NP is whether we can solve all the problems we want to solve, whether we can know everything we want to know. This is basically about the limits of our knowledge. You can think there is no limit to what we may want to know but I claim that essentially in any human endeavor, what people are searching for have the property that when a solution is found, it is easily recognized. NP are all problems that we can honestly say we really want to solve and P are those that we can solve. Whether they are equal is just the very fact that a solution can be easily recognized would imply that it can be efficiently found. That means that we can know everything we ever want to know, it's a fundamental question about human knowledge."

"What is P vs NP and how does it relate to human knowledge?" Avi Wigderson (Turing Award & Abel Prize winner): "In a very basic way, all the problems we want to solve are really NP problems. A subset of it is the problems we can solve. So the question of P versus NP is whether we can solve all the problems we want to solve, whether we can know everything we want to know. This is basically about the limits of our knowledge. You can think there is no limit to what we may want to know but I claim that essentially in any human endeavor, what people are searching for have the property that when a solution is found, it is easily recognized. NP are all problems that we can honestly say we really want to solve and P are those that we can solve. Whether they are equal is just the very fact that a solution can be easily recognized would imply that it can be efficiently found. That means that we can know everything we ever want to know, it's a fundamental question about human knowledge."

Ryan Peterman

20,069 views • 4 months ago

Imagine what we could accomplish in this country, imagine the new businesses we would build, if we didn't have to worry about health insurance. Imagine how much less anxiety and suffering. Medicare for All is an economic obligation, and a moral one.

Imagine what we could accomplish in this country, imagine the new businesses we would build, if we didn't have to worry about health insurance. Imagine how much less anxiety and suffering. Medicare for All is an economic obligation, and a moral one.

Graham Platner for Senate

19,330 views • 5 months ago

The easiest way to build an EV is to buy parts from a catalog and screw them together. We decided to do it the hard way. This is what happens when you build an entire ecosystem from scratch. Over 100 granted patents working as one machine. We at Matter built our own dual cradle frame. We engineered a custom motor and gearbox. We designed active thermal cooling and a smart battery brain so performance never drops. It took years of obsession to get this right. We didn’t assemble this machine. We engineered it.

The easiest way to build an EV is to buy parts from a catalog and screw them together. We decided to do it the hard way. This is what happens when you build an entire ecosystem from scratch. Over 100 granted patents working as one machine. We at Matter built our own dual cradle frame. We engineered a custom motor and gearbox. We designed active thermal cooling and a smart battery brain so performance never drops. It took years of obsession to get this right. We didn’t assemble this machine. We engineered it.

Mohal Lalbhai

453,001 views • 4 months ago

Jensen Huang open-sourced NVIDIA's flagship AI model, its weights, its data, AND how they created it. "We open sourced the models," Huang says. "We open sourced the weights." "We open sourced the data." "We open sourced how we created it." Four layers of openness in one model release. "Open source is fundamentally necessary for many industries to join the AI revolution." "NVIDIA has the scale and the motivation to build and continue to build these AI models for as long as we shall live." The chip seller benefits when every model is open. NVIDIA is the chip seller. P.S. I made a playbook breaking down 100+ most powerful decision making mental models used by history's greatest thinkers. 5,000+ downloads. 113 five-star reviews. Grab a free copy here: If you're new here, follow GeniusThinking for content on the greatest minds in economics, psychology, and history. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

Jensen Huang open-sourced NVIDIA's flagship AI model, its weights, its data, AND how they created it. "We open sourced the models," Huang says. "We open sourced the weights." "We open sourced the data." "We open sourced how we created it." Four layers of openness in one model release. "Open source is fundamentally necessary for many industries to join the AI revolution." "NVIDIA has the scale and the motivation to build and continue to build these AI models for as long as we shall live." The chip seller benefits when every model is open. NVIDIA is the chip seller. P.S. I made a playbook breaking down 100+ most powerful decision making mental models used by history's greatest thinkers. 5,000+ downloads. 113 five-star reviews. Grab a free copy here: If you're new here, follow GeniusThinking for content on the greatest minds in economics, psychology, and history. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

GeniusThinking

14,769 views • 4 months ago

🔥Thomas Massie: "I decided before today, we were going to win or lose today —but we would win either way." "We won because we started a movement." "What happened today was God's will. And we have to figure out what was the purpose of having the biggest fight ever?" "Why did it converge on one of 435 congrsssional seats right here in Kentucky? What was God's purpose? What is He showing us tonight?" 🔥"We're just getting started." 🇺🇸

🔥Thomas Massie: "I decided before today, we were going to win or lose today —but we would win either way." "We won because we started a movement." "What happened today was God's will. And we have to figure out what was the purpose of having the biggest fight ever?" "Why did it converge on one of 435 congrsssional seats right here in Kentucky? What was God's purpose? What is He showing us tonight?" 🔥"We're just getting started." 🇺🇸

Louisa Clary

133,943 views • 4 months ago

In 2022, when we said we wanted to build giant satellites, people told us “No one is ever going to need satellites that large” and “Building satellites that large would be really difficult.” When we said that we wanted to call our company K2, for a Type 2 Kardashev civilization, people asked us “What is a Kardashev?” Fast forward four years and the Kardashev scale is now a mainstream concept, the entire market is moving bigger, and we just completed 100 days on orbit for our first giant satellite mission. This is the story of Gravitas, from First Round

In 2022, when we said we wanted to build giant satellites, people told us “No one is ever going to need satellites that large” and “Building satellites that large would be really difficult.” When we said that we wanted to call our company K2, for a Type 2 Kardashev civilization, people asked us “What is a Kardashev?” Fast forward four years and the Kardashev scale is now a mainstream concept, the entire market is moving bigger, and we just completed 100 days on orbit for our first giant satellite mission. This is the story of Gravitas, from First Round

Karan Kunjur

62,069 views • 2 months ago

Tomi Lahren asks Dana White why the UFC doesn’t do Pride Nights. His answer is a masterclass in how you stay civil while giving zero f*cks about caving to the mob. LAHREN: “Why doesn’t the UFC have Pride Nights?” WHITE: “Listen… I’m assuming we have some gay fighters. I don’t know how many… I know that we have gay female fighters.” “I don’t care what you are, who you are, what you do. We don’t talk about any of that stuff.” “It’s not like we haven’t supported the gay community in the past. We don’t shove any of that stuff down anybody’s throats.” “When we do what we do, we do it because WE WANT TO DO IT. We don’t do it because we want to impress anybody.” “How about you just be honest and real about who you are and what you are as a company? How about that?”

Tomi Lahren asks Dana White why the UFC doesn’t do Pride Nights. His answer is a masterclass in how you stay civil while giving zero f*cks about caving to the mob. LAHREN: “Why doesn’t the UFC have Pride Nights?” WHITE: “Listen… I’m assuming we have some gay fighters. I don’t know how many… I know that we have gay female fighters.” “I don’t care what you are, who you are, what you do. We don’t talk about any of that stuff.” “It’s not like we haven’t supported the gay community in the past. We don’t shove any of that stuff down anybody’s throats.” “When we do what we do, we do it because WE WANT TO DO IT. We don’t do it because we want to impress anybody.” “How about you just be honest and real about who you are and what you are as a company? How about that?”

The Vigilant Fox 🦊

1,664,404 views • 3 months ago

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

MTS

44,057 views • 4 months ago

The biggest bet that I am taking is in the Hyperliquid Ecosystem as HIP-3 takes over and disrupts the financial market: $PURR You need to ask hard questions when you are working through an investment thesis and this is exactly what we did in the livestream today. We went through all of the most significant risks and WHY $PURR is an exceptional opportunity in spite of the short-term price action We sat down with the CEO of $PURR to reanchor the market to WHAT is happening, WHY it matters, and HOW to capitalize on it

The biggest bet that I am taking is in the Hyperliquid Ecosystem as HIP-3 takes over and disrupts the financial market: $PURR You need to ask hard questions when you are working through an investment thesis and this is exactly what we did in the livestream today. We went through all of the most significant risks and WHY $PURR is an exceptional opportunity in spite of the short-term price action We sat down with the CEO of $PURR to reanchor the market to WHAT is happening, WHY it matters, and HOW to capitalize on it

Capital Flows

20,858 views • 3 months ago

Metropolis CEO Alexander Israel on raising $1.6B in debt to acquire and rollup legacy businesses and improve them with AI and technology: "We had unit economic fit. We had product market fit. But what we didn't have was go to market fit. We just hit a wall when we talked to customers. They didn't want to take the risk of working with a startup. So we acquired an existing parking lot operator to deploy our technology across the portfolio. And we doubled the gross profit overnight. We realized that instead of this ground war of organic sales, we could scale very quickly organically. So we went out and acquired another one. It was 10x bigger, a publicly traded company and the largest player in the space."

Metropolis CEO Alexander Israel on raising $1.6B in debt to acquire and rollup legacy businesses and improve them with AI and technology: "We had unit economic fit. We had product market fit. But what we didn't have was go to market fit. We just hit a wall when we talked to customers. They didn't want to take the risk of working with a startup. So we acquired an existing parking lot operator to deploy our technology across the portfolio. And we doubled the gross profit overnight. We realized that instead of this ground war of organic sales, we could scale very quickly organically. So we went out and acquired another one. It was 10x bigger, a publicly traded company and the largest player in the space."

Turner Novak 🍌🧢

11,971 views • 5 months ago

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,935 views • 4 months ago