Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Extropic founder Beff (e/acc) explains how Z1 chips sitting alongside GPUs could make AI data centers 100X more energy efficient: "The reason this hasn't been really explored is that there's not a huge upside for sparsity on traditional accelerators. Because GPUs aren't really good if you try to do... matmuls that are sparse." "It's like shipping a big freight ship across the ocean with a bunch of empty containers. You're wasting a lot of gas. Whereas for us it's compute and memory, it's naturally sparse. We get a huge energy efficiency gain there" "Nowadays everybody's obsessed with tokens per watt at a data center level. If you incorporate TSUs like Z1 into your data center and park them next to your GPUs, you can get 100X energy efficiency gain at a data center level." "People are struggling to power these data centers. It's become the bottleneck. There's a bunch of GPUs sitting on the sidelines right now that can't be powered on, and people are thinking of creating a Dyson swarm to get more power. Why don't we just compute more efficiently first?" Extropicshow more

MTS

568,075 subscribers

16,432 views • 1 month ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

4 Comments

Dillon Mooney1 month ago

@beffjezos @grok True?

Ricci Research1 month ago

The freight analogy cuts the other way too: a ship that only sails when it has the right cargo is efficient per container and useless as a scheduled service. Thermodynamic sampling wins on energy for workloads that are natively probabilistic, but "100x at the data center level" quietly assumes a mix where those workloads are a large share of the rack — and today's mix is dense transformer matmuls, which is precisely the cargo the Z1 can't carry.

Ehsan Azish1 month ago

@beffjezos sparse hardware sitting idle next to power hungry GPUs is such an odd current snapshot of this industry

needmorewatts1 month ago

@beffjezos 100x is a claim, not a grid connection. Show watts per token at useful utilization plus the thermal and interconnect footprint; that’s what operators can buy.

Related Videos

FULL INTERVIEW: Beff (e/acc) says Z1T hits up to 140x the energy efficiency of a GPU, and 100x at the data center level. The end game, he says, is chips in space powered by the sun. Beff (e/acc) is founder and CEO of Extropic. He joined sof 𓋹 to launch Z1T, the first family of transformer-like models built for sparse probabilistic hardware: 01:12 what Z1T is, and how long they've been cooking it 02:32 why models on GPUs are dense and his chip isn't 03:15 a new scaling law where you scale connectivity, not just flops 04:08 how you get 140x, and why a pbit uses 10,000 fewer transistors 07:12 where Z1 sits next to GPUs, and why it's decode not prefill 08:37 100x energy efficiency at the data center level 09:14 you might see Extropic chips in space sooner rather than later 09:45 open weights vs an open training recipe 10:24 Z1T is barely GPT-2, and they started a run last night for GPT-3 11:05 the chips are cooking right now at a major fab overseas 12:58 gated convolutional attention, and the layer they haven't cracked 14:03 the Hardware Lottery, and why he wants people gambling again 15:20 what happens if inference gets 1,000x more efficient 16:59 why you can do this research without a basement full of GPUs

FULL INTERVIEW: Beff (e/acc) says Z1T hits up to 140x the energy efficiency of a GPU, and 100x at the data center level. The end game, he says, is chips in space powered by the sun. Beff (e/acc) is founder and CEO of Extropic. He joined sof 𓋹 to launch Z1T, the first family of transformer-like models built for sparse probabilistic hardware: 01:12 what Z1T is, and how long they've been cooking it 02:32 why models on GPUs are dense and his chip isn't 03:15 a new scaling law where you scale connectivity, not just flops 04:08 how you get 140x, and why a pbit uses 10,000 fewer transistors 07:12 where Z1 sits next to GPUs, and why it's decode not prefill 08:37 100x energy efficiency at the data center level 09:14 you might see Extropic chips in space sooner rather than later 09:45 open weights vs an open training recipe 10:24 Z1T is barely GPT-2, and they started a run last night for GPT-3 11:05 the chips are cooking right now at a major fab overseas 12:58 gated convolutional attention, and the layer they haven't cracked 14:03 the Hardware Lottery, and why he wants people gambling again 15:20 what happens if inference gets 1,000x more efficient 16:59 why you can do this research without a basement full of GPUs

MTS

30,878 views • 1 month ago

Former CEO of Intel, Pat Gelsinger, sees more AI data center delays coming due to lack of energy capacity: "We just need more capacity, and fundamentally, energy capacity omewhat dampens how you fork we can get on AI. Because, you know, why build a new data center and buy the million GPUs if I can't power them? And I think you're gonna see more and more defaults happening on many of those data center projects because the energy won't be there. I think it will be a headwind and I think you're already starting to see some of the first indicators of that with Oracle...Now, it's not like we're gonna slow down as a result, but I think people will start saying, do I really have the energy to underwrite the capital commitments I am making for cement, data center racks, and GPU purchases." Via a16z $GOOGL $META $ORCL $MSFT $AMZN

Former CEO of Intel, Pat Gelsinger, sees more AI data center delays coming due to lack of energy capacity: "We just need more capacity, and fundamentally, energy capacity omewhat dampens how you fork we can get on AI. Because, you know, why build a new data center and buy the million GPUs if I can't power them? And I think you're gonna see more and more defaults happening on many of those data center projects because the energy won't be there. I think it will be a headwind and I think you're already starting to see some of the first indicators of that with Oracle...Now, it's not like we're gonna slow down as a result, but I think people will start saying, do I really have the energy to underwrite the capital commitments I am making for cement, data center racks, and GPU purchases." Via a16z $GOOGL $META $ORCL $MSFT $AMZN

P Equity Research 📰

17,824 views • 18 days ago

Introducing Z1T: Our first family of transformer-like models made for sparse probabilistic hardware like Z1 Achieving up to 140x energy efficiency gains over GPUs and revealing a new scaling law for sparse transformers Read the blog:

Introducing Z1T: Our first family of transformer-like models made for sparse probabilistic hardware like Z1 Achieving up to 140x energy efficiency gains over GPUs and revealing a new scaling law for sparse transformers Read the blog:

Extropic

446,390 views • 1 month ago

Chris Camillo explains how Bloom Energy is solving AI data center energy usage problems… “ $BE is one of the only companies in the world that has a method to essentially bring almost an infinite amount of energy to these data centers." And we won't get into all the technology, but they are now working with hyperscalers and they are a resource for energy that will allow a data center to get ramped up quicker. "So, even though it costs a little bit more money today to put an energy cell from Bloom, an energy box from Bloom, which are energy cells, it essentially takes things like natural gas, but instead of burning the natural gas, there is a chemical process that turns that natural gas into energy for the data center."

Chris Camillo explains how Bloom Energy is solving AI data center energy usage problems… “ $BE is one of the only companies in the world that has a method to essentially bring almost an infinite amount of energy to these data centers." And we won't get into all the technology, but they are now working with hyperscalers and they are a resource for energy that will allow a data center to get ramped up quicker. "So, even though it costs a little bit more money today to put an energy cell from Bloom, an energy box from Bloom, which are energy cells, it essentially takes things like natural gas, but instead of burning the natural gas, there is a chemical process that turns that natural gas into energy for the data center."

The Iced Coffee Hour

277,703 views • 5 months ago

.Doxxnet founder Barrett Lyon on data center myths versus reality: jobs, talent, and faster internet. "That's the other thing that's driving me crazy, is people don't understand data centers at all... Data centers have been around for a very, very long time." "People are being manipulated with this data center thing... It's very interesting to be in the know and then watch this and go, oh my goodness, these people are missing the mark." "The big data center park in Virginia, you just see bunny rabbits in the field. There's nothing. It's just a big building and a bunch of parking lot and then nothing." "It brings jobs. The people that work in them aren't getting paid pennies. And it also brings high tech talent. And then also infrastructure, and infrastructure powers everything you do. So if you have a data center nearby, your internet's faster. Things like that work better." Barrett Lyon Joel de la Garza

.Doxxnet founder Barrett Lyon on data center myths versus reality: jobs, talent, and faster internet. "That's the other thing that's driving me crazy, is people don't understand data centers at all... Data centers have been around for a very, very long time." "People are being manipulated with this data center thing... It's very interesting to be in the know and then watch this and go, oh my goodness, these people are missing the mark." "The big data center park in Virginia, you just see bunny rabbits in the field. There's nothing. It's just a big building and a bunch of parking lot and then nothing." "It brings jobs. The people that work in them aren't getting paid pennies. And it also brings high tech talent. And then also infrastructure, and infrastructure powers everything you do. So if you have a data center nearby, your internet's faster. Things like that work better." Barrett Lyon Joel de la Garza

a16z

35,044 views • 27 days ago

.Heron Power Electronics CEO Drew Baglino on the 40-year streak that kept US electricity flat while the economy grew, and why it just ended: "We benefited from energy efficiency from the '70s... until like the 2010s. You saw efficiency everywhere... And all of that energy efficiency more or less negated the growth in energy services." "The energy services were growing over time in the United States, but the net electricity delivered to accomplish those energy services stayed basically flat. And the same thing even happened in data centers." "If you look at the efficiency of compute, even with all the AI things that are going on, in terms of flops per watt, it's really gone up an immense amount from the first data centers to today." "And that has allowed more and more compute capability for the same percentage of GDP electricity production." "Now it's changing... We are seeing electricity growth higher than it's ever been before. And that's not just because the benefits of energy efficiency are tailing off, it's because we are seeing broad based demand for more electricity: compute, transportation, industrial electrification." "Electricity as a carrier is incredibly flexible... whether it's for intelligence or for heating, I see it as a great thing and wanna accelerate it." Drew Baglino Heron Power Electronics

.Heron Power Electronics CEO Drew Baglino on the 40-year streak that kept US electricity flat while the economy grew, and why it just ended: "We benefited from energy efficiency from the '70s... until like the 2010s. You saw efficiency everywhere... And all of that energy efficiency more or less negated the growth in energy services." "The energy services were growing over time in the United States, but the net electricity delivered to accomplish those energy services stayed basically flat. And the same thing even happened in data centers." "If you look at the efficiency of compute, even with all the AI things that are going on, in terms of flops per watt, it's really gone up an immense amount from the first data centers to today." "And that has allowed more and more compute capability for the same percentage of GDP electricity production." "Now it's changing... We are seeing electricity growth higher than it's ever been before. And that's not just because the benefits of energy efficiency are tailing off, it's because we are seeing broad based demand for more electricity: compute, transportation, industrial electrification." "Electricity as a carrier is incredibly flexible... whether it's for intelligence or for heating, I see it as a great thing and wanna accelerate it." Drew Baglino Heron Power Electronics

a16z

35,943 views • 17 days ago

.Beff (e/acc) explains how thermodynamic computing fits into the current computing stack: "We don't do GPUs. They're called TSUs, thermodynamic sampling units. We think people are gonna put them next to GPUs or whatever their favorite accelerator is." "You don't need to run the whole workload on the TSU, but it could run certain parts of the workload, just like the GPU is complementary to the CPU initially." "The TSU is gonna help give you more performance per watt in your data center when combined with existing build-outs." "Let's say you're serving a large diffusion model. What we showed is that by using a TSU, you can use something called an energy-based model instead of a simple probabilistic kernel." "Overall you can have basically 100 times the performance per watt of just running on pure GPUs because our chips are much lower wattage."

.Beff (e/acc) explains how thermodynamic computing fits into the current computing stack: "We don't do GPUs. They're called TSUs, thermodynamic sampling units. We think people are gonna put them next to GPUs or whatever their favorite accelerator is." "You don't need to run the whole workload on the TSU, but it could run certain parts of the workload, just like the GPU is complementary to the CPU initially." "The TSU is gonna help give you more performance per watt in your data center when combined with existing build-outs." "Let's say you're serving a large diffusion model. What we showed is that by using a TSU, you can use something called an energy-based model instead of a simple probabilistic kernel." "Overall you can have basically 100 times the performance per watt of just running on pure GPUs because our chips are much lower wattage."

MTS

42,570 views • 5 months ago

🚨 JUST IN: JD Vance helps to clarify President Trump's stance on data centers, highlighting his desire for data centers to put power BACK into the grid "I think probably 99% of the backlash to data centers has come in areas where building a data center means higher utility and higher electricity for the people on the ground." "If you build a data center, you should be putting power back into the grid, not taking it out, and if that is happening, I don't think the data centers are that controversial."

🚨 JUST IN: JD Vance helps to clarify President Trump's stance on data centers, highlighting his desire for data centers to put power BACK into the grid "I think probably 99% of the backlash to data centers has come in areas where building a data center means higher utility and higher electricity for the people on the ground." "If you build a data center, you should be putting power back into the grid, not taking it out, and if that is happening, I don't think the data centers are that controversial."

Nick Sortor

217,808 views • 1 month ago

Lutnick: If you are building a data center, you were going to give power to the grid and drive the energy costs of everybody around that data center lower. Trump is going to bring lower energy prices. Everywhere that a big company builds a big data center, they are going to lower the price of energy in the entire area around it.

Lutnick: If you are building a data center, you were going to give power to the grid and drive the energy costs of everybody around that data center lower. Trump is going to bring lower energy prices. Everywhere that a big company builds a big data center, they are going to lower the price of energy in the entire area around it.

Acyn

494,248 views • 10 months ago

Flapping Airplanes co-founder Asher Spector explains why data efficiency is the greatest bottleneck to AI adoption: "To the extent that AI has been hard to integrate into the economy, I really think it's because models are much less data-efficient than humans. If you want it to learn a new task, or put it in a new vertical, it takes thousands of times more effort than it does to just tell a human what to do." "If you can make a model a million times more data-efficient, it's a million times easier to put into the economy. There's a ton of cool stuff that you can do in really data-constrained regimes. For example, whether it's robotics, or scientific discovery, or even something like trading, these problems have very limited data, and existing AI systems aren't quite as good at them as they are at other things. I think that learning to learn with less data is just tremendously valuable in all of these domains."

Flapping Airplanes co-founder Asher Spector explains why data efficiency is the greatest bottleneck to AI adoption: "To the extent that AI has been hard to integrate into the economy, I really think it's because models are much less data-efficient than humans. If you want it to learn a new task, or put it in a new vertical, it takes thousands of times more effort than it does to just tell a human what to do." "If you can make a model a million times more data-efficient, it's a million times easier to put into the economy. There's a ton of cool stuff that you can do in really data-constrained regimes. For example, whether it's robotics, or scientific discovery, or even something like trading, these problems have very limited data, and existing AI systems aren't quite as good at them as they are at other things. I think that learning to learn with less data is just tremendously valuable in all of these domains."

TBPN

92,663 views • 9 months ago

MARK CUBAN: “We're building these data centers and if there's a price performance curve on AI that minimizes the power requirements, there's going to be a lot of data centers that are going to be turned into pickleball courts.”

MARK CUBAN: “We're building these data centers and if there's a price performance curve on AI that minimizes the power requirements, there's going to be a lot of data centers that are going to be turned into pickleball courts.”

Chief Nerd

58,468 views • 3 months ago

MoistCr1TiKaL comes out strongly against data centers in recent video and ponders if they're making people sicker due to hoarding clean water "AI data centers are actually the boogeyman..a scourge on this planet and a stain on civilized society..no normal ppl with functioning frontal lobes want data centers in their community, it is bad for them" "I've been getting conspiratorial..I was reading something that someone was sharing about like when a AI data center came through, and the impact it had on their water, cause that's a very common thing" "I'm not fully bought into it..but it's an interesting theory..what if the reason there's been seemingly so much more sickness, like it feels like ppl are getting sick more frequently now and a wide variety of illnesses just seem more common..what if AI data centers are hoarding all of the clean water..what if all the clean water is being diverted to the AI data centers and the normal civilians are just getting the sludge runoff..I don't think it's out of the realm of possibility..they are evil enough to do that" (Via penguinz0 on YT)

MoistCr1TiKaL comes out strongly against data centers in recent video and ponders if they're making people sicker due to hoarding clean water "AI data centers are actually the boogeyman..a scourge on this planet and a stain on civilized society..no normal ppl with functioning frontal lobes want data centers in their community, it is bad for them" "I've been getting conspiratorial..I was reading something that someone was sharing about like when a AI data center came through, and the impact it had on their water, cause that's a very common thing" "I'm not fully bought into it..but it's an interesting theory..what if the reason there's been seemingly so much more sickness, like it feels like ppl are getting sick more frequently now and a wide variety of illnesses just seem more common..what if AI data centers are hoarding all of the clean water..what if all the clean water is being diverted to the AI data centers and the normal civilians are just getting the sludge runoff..I don't think it's out of the realm of possibility..they are evil enough to do that" (Via penguinz0 on YT)

yeet

3,070,902 views • 3 months ago

If chips, models, networks & agent architectures all become dramatically more efficient.. Do we still really need $50-100 Billion data centers? "I think you're gonna see this new wave of companies that are doing distributed data centers. These data centers are mid-size. It's gonna be even more important as you go into this agentic world because you're not dealing with a single model and a single prompt. These agents are orchestrating with each other."

If chips, models, networks & agent architectures all become dramatically more efficient.. Do we still really need $50-100 Billion data centers? "I think you're gonna see this new wave of companies that are doing distributed data centers. These data centers are mid-size. It's gonna be even more important as you go into this agentic world because you're not dealing with a single model and a single prompt. These agents are orchestrating with each other."

Molly O’Shea

38,044 views • 3 months ago

Hassett: "Data centers are very good for towns, because they create so many jobs and bring people in with high incomes that can buy houses and stuff like that. So if you take a sleepy town that hasn't seen much in the last 30 years and put a data center there, there are gonna be a whole bunch of happy residents."

Hassett: "Data centers are very good for towns, because they create so many jobs and bring people in with high incomes that can buy houses and stuff like that. So if you take a sleepy town that hasn't seen much in the last 30 years and put a data center there, there are gonna be a whole bunch of happy residents."

Aaron Rupar

1,104,339 views • 2 months ago

Yann LeCun -- Meta's chief AI scientist and a Turing Award winner -- explains why scaling up LLMs will never reach human-level AI, and names the four things today's systems still can't do: "If you think that we're going to get to human-level AI by just training on more data and scaling up LLMs, you're making a mistake." "If you're an investor and you invest in a company that told you we're going to get to human-level AI and PhD level by just training on more data and with a few tricks... that was probably not a good idea." "There are ideas about how to go forward and have systems that are capable of doing what every intelligent animal and human are capable of doing, and that current AI systems are not capable of doing." "Understanding the physical world, having persistent memory, and being able to reason and plan. Those are the four characteristics that need to be there." "That requires systems that can acquire common sense, that can learn from natural sensors like video, as opposed to just text." The entire market is priced on a straight line from bigger models to AGI. One of the people who invented deep learning is telling you the line doesn't reach -- and that the real bottleneck isn't compute, it's world models. If he's right, the winner isn't whoever stacks the most GPUs. It's whoever teaches a machine to learn from video the way a child does.

Yann LeCun -- Meta's chief AI scientist and a Turing Award winner -- explains why scaling up LLMs will never reach human-level AI, and names the four things today's systems still can't do: "If you think that we're going to get to human-level AI by just training on more data and scaling up LLMs, you're making a mistake." "If you're an investor and you invest in a company that told you we're going to get to human-level AI and PhD level by just training on more data and with a few tricks... that was probably not a good idea." "There are ideas about how to go forward and have systems that are capable of doing what every intelligent animal and human are capable of doing, and that current AI systems are not capable of doing." "Understanding the physical world, having persistent memory, and being able to reason and plan. Those are the four characteristics that need to be there." "That requires systems that can acquire common sense, that can learn from natural sensors like video, as opposed to just text." The entire market is priced on a straight line from bigger models to AGI. One of the people who invented deep learning is telling you the line doesn't reach -- and that the real bottleneck isn't compute, it's world models. If he's right, the winner isn't whoever stacks the most GPUs. It's whoever teaches a machine to learn from video the way a child does.

Karl Mehta

34,860 views • 3 months ago

"A data center is one of the absolute worst economic development tools that you can have in a community. It doesn't create any jobs. It doesn't create any knock-on economic effects, because the only thing that wants to be located next to a data center is a data center." It's time to ban tax breaks for data centers. Pat_Garofalo breaks it down on MS NOW:

"A data center is one of the absolute worst economic development tools that you can have in a community. It doesn't create any jobs. It doesn't create any knock-on economic effects, because the only thing that wants to be located next to a data center is a data center." It's time to ban tax breaks for data centers. Pat_Garofalo breaks it down on MS NOW:

American Economic Liberties Project

96,128 views • 2 months ago

ASML CEO on Elon Musk's Data Centers in space and Terafab: "I think you need a certain amount of data centers and they will be in space, they will be on the earth. I think the discussion about data centers in space is to address another potential bottleneck in the ecosystem, which is energy. I think the discussion around putting data center in space is there because if you do that, the energy consumption or the energy availability will be there, about trying to solve another potential big issue, which is energy for data center. I think the Terafab is an example of a major fab project."

ASML CEO on Elon Musk's Data Centers in space and Terafab: "I think you need a certain amount of data centers and they will be in space, they will be on the earth. I think the discussion about data centers in space is to address another potential bottleneck in the ecosystem, which is energy. I think the discussion around putting data center in space is there because if you do that, the energy consumption or the energy availability will be there, about trying to solve another potential big issue, which is energy for data center. I think the Terafab is an example of a major fab project."

DogeDesigner

48,139 views • 4 months ago

Gavin Baker: "if you're a foundation model company and you do not have unique data and internet scale distribution, you are the fastest depreciating asset in human history - I think most of these companies are zeros, and there's like 10 of them" this is him explaining why the 100 largest buildings in the world will be data centers, why there are only three places left in America you can legally build one, and what he says is setting up the first real memory cycle since 1996 "in our lifetime there will be data centers that are bigger than Manhattan" "the two rate limiting factors for AI are no longer compute and GPUs - it's power, finding places to plug the GPUs in" "there's only three places in the United States you can put one of these - it's got to be near multiple nuclear reactors that are unregulated within 50 miles of each other" "if Apple doubles the memory in the iPhone and Google does the same for Android at the same time, you could have the first real memory cycle since 1996 - and in 1996 Micron was a 50 bagger" "the only hope most of these companies have is for Amazon to acquire them the way Microsoft did with Inflection" bookmark & watch the full conversation ↓

Gavin Baker: "if you're a foundation model company and you do not have unique data and internet scale distribution, you are the fastest depreciating asset in human history - I think most of these companies are zeros, and there's like 10 of them" this is him explaining why the 100 largest buildings in the world will be data centers, why there are only three places left in America you can legally build one, and what he says is setting up the first real memory cycle since 1996 "in our lifetime there will be data centers that are bigger than Manhattan" "the two rate limiting factors for AI are no longer compute and GPUs - it's power, finding places to plug the GPUs in" "there's only three places in the United States you can put one of these - it's got to be near multiple nuclear reactors that are unregulated within 50 miles of each other" "if Apple doubles the memory in the iPhone and Google does the same for Android at the same time, you could have the first real memory cycle since 1996 - and in 1996 Micron was a 50 bagger" "the only hope most of these companies have is for Amazon to acquire them the way Microsoft did with Inflection" bookmark & watch the full conversation ↓

Сarm1ne

274,201 views • 2 months ago