Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Gavin Baker (Gavin Baker) says the disaggregation of inference can extend GPU useful lives from 3-4 years to 10-15. That may single-handedly save private credit and reduce the financing rates for GPUs, which will drive demand and help finance the build-out. "The disaggregation of prefill and inference is going...

206,845 Aufrufe • vor 1 Monat •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

CoreWeave $CRWV CEO Michael Intrator, on the Big Technology Podcast, providing what I believe is the most compelling rebuttal to the narrative that GPUs only have a 2 to 3-year useful life. CoreWeave's customers are the world's most sophisticated users of GPUs, and they are signing contracts to utilize today's GPUs for the next 5 years, despite Nvidia's new product release cadence resulting in more efficient GPUs every 1-2 years. The use-case for the GPUs might change over time as Nvidia rolls out new products (e.g. older chips are moved from training to inference workloads), but the "usefulness" does not. "The most important tool that I have for understanding what the depreciation curve or the obsolescence curve of compute is not what I think, right? It's not what some historic short (seller) thinks. It's what are the buyers, the most sophisticated companies in the world are willing to pay for today. And when they come to me and they put in a contract for a 5-year deal or 6-year deal, in what world do I not think that they who are the consumers of this understand that there are new, more powerful chips coming out? Of course they do. They understand it, but they also understand what their various use cases are. And they're saying to themselves, "I'm going to buy this because I'm going to need it today. I'm going to need it in 3 years, and I'm going to need it in 5 years"... My opinions around depreciation are informed by the only entities that get to vote in my world, which are the folks that are paying for the compute over time."

Rittenhouse Research

161,641 Aufrufe • vor 5 Monaten

David Friedberg: Michael Burry’s Datacenter Math is Wrong “I actually think Michael Burberry's got this wrong.” “What Michael Burry is saying is that all of these hyperscalers have extended their depreciation schedule or the useful life of their data centers by roughly 2x, which cuts the operating costs in half when they report it in earnings. And so it's making their earnings inflate.” “So he's claiming they're cooking the books. Google first made this change in Q1 of 2021, where they said the servers are now going from 3 to 4 years. Separately in 2021, Google took networking equipment from 3 to 5 years. And then in 2023, they took it from 5 to 6 years.” “And so this is a result of this effort where they went in and did an analysis. So what happened?” “What happened in the data centers is that the data centers transitioned from being primarily data storage and data transfer systems, where you would use hard drives and RAM and memory to store data and then transmit it back out, to being data processing centers because of the AI boom.” “So as AI became more important in the data center, more of the dollars that are going into data centers were allocated towards chips from data storage, which initially was hard drives.” “And then suddenly, when you put these processors in to process the data to do AI, the majority of the spend and the majority of the energy is going towards the processors.” “I made some calls and I checked around with some other friends, and everyone says the same thing: that these 7-8 year old TPUs and GPUs that are sitting in the data centers are still being used and they're being used at 100% utilization.” “So that actually justifies and validates the depreciation schedule being much longer versus shorter.”

The All-In Podcast

304,297 Aufrufe • vor 7 Monaten

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 Aufrufe • vor 5 Tagen

In just 3 minutes, Ken Griffin (Citadel) & Larry Fink (BlackRock) explain the current state of government overspending, AI hype, & 2026 $600B data center CapEx "The world needs a savior, & the hope is that AI is the savior that we need for productivity." "The the area of recklessness is the spending of governments around the world, who are all, with little exception, all spending well beyond their means." Big issue: "Will AI create the productivity acceleration that is honestly just hoped for in Washington & in the halls of government around the world as a ways to overcome the profligate spending that we're currently engaged in?" Citadel BlackRock World Economic Forum . . . Ken Griffin "Let's take a step back and and talk about where we are right here right now. The the area of recklessness is the spending of governments around the world, who are all, with little exception, all spending well beyond their means. That's the recklessness of this moment in history. This is not a parallel to the 1920s in terms of the recklessness of the of the private capital markets. It's a story of the recklessness of government spending. Within the private sector there's a huge question as to where AI will take us. And I, I was carefully taking notes and listening to what Larry has to say, or to what Madame Lagarde has to say, because this is one of the big issues of our moment. Will AI create the productivity acceleration that is honestly just hoped for in Washington and in the halls of government around the world as a ways to overcome the profligate spending that we're currently engaged in? The world, the world needs a savior, and the hope is that AI is the savior that we need for productivity. And the challenge with this is it is, it may or may not be. We just don't know yet. Now there's a tremendous amount of hype around AI, and in some sense the large AI companies need to create that hype to raise the tens — or actually 100, hundreds of billions, right — of billions of dollars of investment that are going into the field. Like you wouldn't be able to raise hundreds of billions of dollars. We'll spend — and Larry can probably correct me on this — but roughly $600 billion this year in cap ex for data centers in the United States. Larry Fink "I think it could be larger." Andrew Ross Sorkin "But does that mean that it's getting hyped up too much, or it's just the the hype is required as a sales mechanism?" Larry Fink "First of all, so much of the data centers are being built for cloud, right? And and the big issue is gonna be in terms of monetization of of of the spend. The data centers are being built for AI requires more advanced chips. The question is what is the lifetime of that chip. If we have new technological changes in the lifetime of the chip in one year, then that spend is gonna be really a bad spend. If the lifetime as they expect it to be is 4 or 5 years and then those chips can be used for cloud, then then I think these investments are gonna prove to be good investments. So I think it's it's gonna be — you know, if the speed of technology changes and all these investments now, they're gonna be — it's gonna be a challenge. But I agree with Ken. I think we don't know enough, but I'm personally very optimistic on how AI is gonna affect the world economy."

Molly O’Shea

33,210 Aufrufe • vor 5 Monaten

We keep hearing that retail Indians are borrowing too much, but despite all the noise, credit penetration in India remains relatively low and a significant population remains credit-starved. That said, to the extent that individual Indians do have credit, I think Bajaj Finance has done more to improve access than even many of the bigger banks. In a lot of ways, they've been pioneers. They took loan approvals from 3–5 days to 3 hours to, eventually, 3 minutes. They are also great at leveraging data. They track how old your appliances are and figure out your refrigerator is about to need replacing before you've even thought about it, and walking into Croma or Vijay Sales with a list of 50,000 customers ready to buy. Bajaj Finance knew you better than your retailer did. Having said that, businesses like Bajaj Finance and even Zerodha have had it relatively easy over the past decade. As India went online and consumers got access to easy credit and the ability to invest easily, we kind of rode that wave. But as the saying goes, success attracts competition. With all the new players now wanting to be lenders, including us (Zerodha Capital ) with our secured business, the future will be interesting, to say the least. Almost done with Episode 2 of The Ken's Intermission, and this one is on the giant that is Bajaj Finance, what it took to build that machine, and what it means for the future of credit in India. It’s free to watch. Link in comments. Disclosure: Zerodha had a small part in the making of Intermission.

Nithin Kamath

52,895 Aufrufe • vor 1 Monat