Loading video...

Video Failed to Load

Go Home

“If AI is technologically possible, it is inevitable.” - Dwarkesh Patel We’re mid-Cambrian Explosion: • 4× annual jump in frontier-model spend • Energy demand will hit nation-state scale by 2028 • Chips & compute racing toward $ 1Tn+ industries This is the Scaling Era—a new normal written in real...

18,823 views • 1 year ago •via X (Twitter)

3 Comments

Limitless's profile picture
Limitless1 year ago

Listen or watch on your favorite platform here:

Na's profile picture
Na1 year ago

@dwarkesh_sp 31:25 models have to feel it in their bones (weights), just like humans tie their identity to a job and develop instuition and familiar routine of working paths. So my guess is there will be countless individually trained models distilled from largest models.

PIN AI's profile picture
PIN AI1 year ago

@dwarkesh_sp AGI seems inevitable, but who it serves is uncertain. The Scaling Era will shape both its abilities and values. PIN AI aims for a future where your AI responds to you, not big tech.

Related Videos

Jonathan Ross just revealed why AI companies aren’t growing faster. Not demand. Not competition. Physics. Ross: “The demand for compute is insatiable.” There isn’t enough compute in the world. Not a temporary shortage. A fundamental gap between what the market wants and what the infrastructure can deliver. Ross: “Right now, one of the biggest complaints of Anthropic is the rate limits. People can’t get enough tokens.” Rate limits aren’t product decisions. They’re rationing. Companies forced to regulate access because infrastructure cannot meet demand. Slower services. Token caps. The only things standing between these companies and a revenue surge they can’t access. Every token cap is a revenue cap. Every slowdown is a sale that didn’t happen. Ross: “If Anthropic was given twice the inference compute, within one month their revenue would almost double.” Read that again. Double the compute. Double the revenue. Within thirty days. That’s not a growth projection. That’s a measurement of how deep the backlog already is. The demand exists right now. It’s sitting in a queue. The only thing between these companies and that revenue is physical hardware they don’t have. This breaks every assumption about how tech companies scale. Usually you scale by finding customers. AI companies have infinite customers. They scale by finding hardware. The constraint isn’t market fit. It isn’t distribution. It isn’t competition. It’s processing power. This is why Jensen Huang is the most important person in the world right now. NVIDIA doesn’t just make chips. It makes the thing every government, every AI lab, and every company racing for this future needs more of and can’t get enough of. The compute bottleneck isn’t a tech industry problem. It’s a civilizational one. The winner of this era isn’t determined by who builds the smartest model. Every major lab has a frontier model. The winner is whoever secures the most compute fastest while everyone else rations what’s left. The race isn’t for intelligence. It’s for infrastructure. And right now there isn’t enough to go around.

Dustin

28,395 views • 4 months ago

If intelligence is the log of compute… it starts with a lot of compute! And that’s why we’re scaling our GPU fleet faster than anyone else. Just last year, we added over 2 gigawatts of new capacity – roughly the output of 2 nuclear power plants. And today we’re going further, announcing the world's most powerful AI datacenter, located in southeastern Wisconsin. Fairwater is a seamless cluster of hundreds of thousands of NVIDIA GB200s, connected by enough fiber to circle the Earth 4.5 times. It will deliver 10x the performance of the world’s fastest supercomputer today, enabling AI training and inference workloads at a level never before seen. For AI training workloads, you need compute at exponential scale. That’s why we designed the datacenter, GPU fleet, and network together as one integrated system. This ensures a single job can run from day 1 at exponential scale across thousands of GPUs. Fairwater uses a liquid-cooled closed-loop system for cooling GPUs that requires zero water for operations after construction. And we’re matching all of the energy that is consumed with renewable sources. And of course, it is just one of several similar sites we’re lighting up across our 70+ regions. We have multiple identical Fairwater datacenters under construction in other locations across the US, in addition to our AI infrastructure already deployed in over 100 datacenters around the world, powering model training, test-time compute, RL tuning, and real-time inference at global scale. Too often during times like this, people go with the current and only later wonder, how did we get here? With Fairwater, we're charting a new path: doing the hard engineering work, bringing compute, network, and storage into one highly scaled cluster, and designing closed-loop energy systems to meet real-world computing needs. And partnering with local communities to ensure it's thoughtfully done in a way that is sustainable, creates new jobs, and expands opportunity. We are thrilled to see this take hold in Wisconsin, and we are just getting started.

Satya Nadella

2,020,601 views • 10 months ago

Distilled recap of the back-and-forth with Jensen on export controls: Dwarkesh: Wouldn’t selling Nvidia chips to China enable them to train models like Claude Mythos with cyber offensive capabilities that would be threats to American companies and national security? Jensen: First of all, Mythos was trained on fairly mundane capacity and a fairly mundane amount of it by an extraordinary company. The amount of capacity and the type of compute it was trained on is abundantly available in China. Dwarkesh: With that, could they eventually train a model like Mythos? Yes. But the question is, because we have more FLOPs, American labs are able to get to this level of capabilities first. Furthermore, even if they trained a model like this, the ability to deploy it at scale matters. If you had a cyber hacker, it's much more dangerous if they have a million of them versus a thousand of them. Jensen: Your premise is just wrong. The fact of the matter is their AI development is going just fine. The best AI researchers in the world, because they are limited in compute, also come up with extremely smart algorithms. DeepSeek is not an inconsequential advance. The day that DeepSeek comes out on Huawei first, that is a horrible outcome for our nation. Dwarkesh: Currently, you can have a model like DeepSeek that can run on any accelerator if it's open source. Why would that stop being the case in the future? Jensen: Suppose it optimizes for Huawei. Suppose it optimizes for their architecture. It would put others at a disadvantage. As AI diffuses out into the rest of the world, their standards and their tech stack will become superior to ours because their models are open. Dwarkesh: Tesla sold extremely good electric vehicles to China for a long time. iPhones are sold in China. They didn't cause some lock-in. China will still make their version of EVs, and they're dominating, or smartphones, they're dominating. Jensen: We are not a car. The fact that I can buy this car brand one day and use another car brand another day is easy. Computing is not like that. There's a reason why x86 still exists. There's a reason why Arm is so sticky. These ecosystems are hard to replace. Dwarkesh: It's just hard to imagine that there's a long-term lock-in to the Chinese ecosystem, even if they have this slightly better open-source model for a while. American labs port across accelerators constantly. Anthropic's models are run on GPUs, they're run on Trainium, they're run on TPUs. There are so many things you can do, from distilling to a model that's well fit for your chips. Jensen: China is the largest contributor to open source software in the world. China's the largest contributor to open models in the world. Today it's built on the American tech stack, Nvidia’s. Fact. All five layers of the tech stack for AI are important. The United States ought to go win all five of them. in a few years time, I'm making you the prediction that when we want American technology to be diffused around the world—out to India, out to the Middle East, out to Africa, out to Southeast Asia—on that day, I will tell you exactly about today's conversation, about how your policy ... caused the United States to concede the second largest market in the world for no good reason at all.

Dwarkesh Patel

1,250,786 views • 3 months ago

Demis Hassabis just explained why the real AI bottleneck has nothing to do with training runs. Most people picture the AI arms race as who can build the biggest model. GPT-4 or Gemini Ultra style training runs, a few hundred million in compute, fired once or twice a year. The constraint sits somewhere else. Every time a researcher has a new algorithmic idea, a new architecture, a new training technique, they can't just test it on a laptop. They have to run it at the scale where it would actually be deployed, because ideas that look promising at small scale fall apart completely when you put them into a real system. Every research hypothesis burns significant compute before a single line of production code gets written. At a lab like DeepMind, hundreds of researchers are running hundreds of ideas simultaneously. The demand for experimental compute is continuous. It never stops. Now layer the hardware reality on top. GPU lead times are currently 36 to 52 weeks for data center hardware. Global AI data centers are already drawing 29.6 gigawatts, equivalent to the peak power demand of the entire state of New York, and they still can't meet demand. Companies willing to pay any price can't just buy more compute. They wait in line. The speed of scientific discovery in AI is now gated by hardware availability. The next breakthrough is sitting in a researcher's head right now. Whether it gets validated fast enough to matter depends entirely on whether the compute is there when they need it. The AI race gets won by whoever can run the most experiments per month.

Aakash Gupta

31,285 views • 2 months ago

Satya is Out TLDR: MSFT doesn’t believe in AGI, wary of overinvestment, OpenAI partnership is over A. AI and AGI are overhyped - “Us self-claiming some AGI milestone, that's just nonsensical benchmark hacking to me. The real benchmark is: the world growing at 10%.” B. Very negative on more capex spend from MSFT - “[If you look at the Industrial Revolution] there was a lot of money lost” - “…countries are going to deploy capital… I'm so excited to be a leaser… I build a lot, I lease a lot.“ C. Value in AI is in infra where there is overbuild, and B2C apps which OpenAI has won, negative on model layer - “in consumer, in some categories, there may be some winner-take-all network effect. ChatGPT is a great example [of a] at-scale consumer property that has already got real escape velocity” - “[in enterprise] buyers will not tolerate winner-take-all. … where the buyer is a corporation, an enterprise, an IT department, they will want multiple suppliers.. That is what will happen even on the model side.” - “In the model layer, one is models need ultimately to run on some hyperscale compute.” Opinion ———— 1. Satya is calling the bubble in buildout. The crazy people like govts are entering the game. He’s happy to lease from them when they overbuild. His own capex spend is capped 2. He’s disappointed in the OpenAI partnership - he sees them as having built a great consumer app for themselves with dominance in the category, but models for enterprise usage have gotten commoditized. 3. He’s sooo done with the AGI talk. If you can’t get to 10% global growth your AGI talk is meaningless to him 4. He’s backing MSFT from the capex precipice. It’s funny because he certainly did make Google dance, and now they’re committed to insane capex. But he’s outta here One of the most meaningful Dwarkesh Patel interviews so far

Prakash

1,447,479 views • 1 year ago

The entire AI industry is racing to build the smartest model. Satya Nadella just admitted that is not where the money is. The model is not the product. The harness is. That is the exact line. And it changes what Microsoft is actually competing on. OpenAI, Anthropic, Google, xAI, Meta every frontier lab is pouring hundreds of billions into training compute, chasing the next capability jump. Each betting that raw model intelligence is the moat. Microsoft is doing the opposite. It is building the harness the orchestration layer that sits above the model, connecting it to tools, data, permissions, sub-agents, and enterprise workflows. And it is letting OpenAI, Anthropic, and MAI compete to plug into it. "You need the model. But the model is not the product. The harness is." So do the math on what a harness actually does. A raw model dropped into an enterprise answers questions. That is a chatbot. A harness turns that same model into an agent that reads the SharePoint, edits the ERP entry, pulls the GitHub PR, updates Salesforce, and files the Excel report with the right permissions, the right audit trail, and the right sub-agent for each sub-task. The model provides the intelligence. The harness converts intelligence into work. Now here's where it gets interesting. "Even the best model in the world will feel broken without a great harness. And an okay model with a great harness can feel like magic." If that is true, the enterprise buyer is not buying model quality. The enterprise buyer is buying the harness. Which means model quality becomes a commodity input over time, and harness quality becomes the sustainable moat. Compare that to the strategy the entire frontier lab industry is executing. Everyone else is chasing the numerator raw intelligence. Almost nobody at scale is racing to build the denominator the orchestration layer that determines whether that intelligence can actually be deployed profitably inside a real company. The frontier model race has a 10 to 20 percent chance of producing a single dominant winner. Nadella just told the industry he does not need to be that winner. If OpenAI wins, Microsoft wins. If Anthropic wins, Microsoft wins. If MAI wins, Microsoft wins. If someone Microsoft has never heard of trains a better model in 2027, Microsoft still wins. Because the compute they train on, the harness they get plugged into, the enterprise contracts they get delivered through, and the products they sit inside are all Microsoft. He is not building the best AI model. He is building the layer that the best AI model has to run on to make anyone money. I wonder which position looks more valuable in ten years.

Vikram M

21,463 views • 12 days ago