Loading video...

Video Failed to Load

Go Home

Agentic workloads generate 20-50x more tokens than chat. That multiplier hits compute, storage, and orchestration simultaneously. Google Cloud's Mark Lohmeyer (who owns all of Compute) and Patrick Moorhead break down how GC is building infrastructure to absorb that pressure across the full AI Hypercomputer stack from Google Cloud Next 2026.

162,434 views • 3 months ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Extra outtake clip from latest Bg2 Pod with Jensen Brad Gerstner Contrary to popular hypersensationalist rhetoric -- that we are in a massive AI glut -- we are likely in a stretch of structural compute shortage. Google announced in May that tokens had grown 50x y/y, and doubled again by July 2025 (100x) to 1 quadrillion monthly tokens. In that period - algorithmic and hardware advances improved efficiencies by ~10-15x - which means Google had to increase accelerated compute dedicated to token generation by 3-10x. Our estimate is that Google increased accelerated compute by ~3x during the period - which means that they had to pull compute from training, recommenders, etc to allocate to token generation. Significant algorithmic advancements (Flash Attention, quantization, MoE), and infrastructure investments (prompt caching, batching) have driven much of that efficiency, but counting on the hardware to get better is something the industry is counting / relying on. We used to be able to ride Moore's Law / Dennard scaling to improve compute per watt. But now... we have to rely on $NVDA / hardware ecosystem (Google, $AMD, etc) to drive 2-4x improvement per generation (Huang's Law). People underestimate the strain of exponentials on human systems - that are hard coded to think linearly... The total global accelerated compute base is probably ~8 GW of installed capacity on my math and analysts have estimated growth to increase to 10x to ~80 GW globally by 2030. Even assuming all of that capex gets done, and Nvidia continues to push yearly roadmap (generating 2x y/y performance uplift), the 10x power increase should equate to ~50x increase in compute. We just had 100x AI usage increase in 1 year from Google's testimony. OpenAI, Google, Anthropic are in structural shortage of compute - each bit they bring online is fully invested in serving their users. And that is without even expanding into true video / world models, robotics, or long horizon thinking to find novel breakthroughs. What could change this trajectory? If the algorithmic efficiencies we are gaining from new breakthroughs outstrip the exponential increase in current demand -- and FUTURE demand. Investors hyperventilated at DeepSeek's release earlier this year, but their gains were outweighed by the increase in demand created by reasoning.

Clark Tang

176,622 views • 10 months ago