Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Argmax now runs on Google Tensor TPU, the first-ever SDK to harness this edge inference accelerator! Tensor TPU enabled us to deploy billion-scale transformers reliably on Pixel phones without impacting battery life or resource contention with traditional workloads.

argmax

4,489 subscribers

27,205 views • 4 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Real-time Transcription with Speakers is now generally available on iOS and macOS! Details for installing or simply testing Argmax SDK 2 are in the comments.

Real-time Transcription with Speakers is now generally available on iOS and macOS! Details for installing or simply testing Argmax SDK 2 are in the comments.

argmax

29,142 views • 6 months ago

Introducing Real-time Transcription with Speakers! - Step change in accuracy, surpassing top cloud APIs - Faster than real-time on Mac and iPhone - Still under 3 watts when all features are enabled Available in Argmax SDK 2.0 for early access! Benchmarks and details in comments.

Introducing Real-time Transcription with Speakers! - Step change in accuracy, surpassing top cloud APIs - Faster than real-time on Mac and iPhone - Still under 3 watts when all features are enabled Available in Argmax SDK 2.0 for early access! Benchmarks and details in comments.

argmax

72,885 views • 8 months ago

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.

argmax

30,154 views • 6 months ago

A masterclass on Google's TPU v8 Networking. Two TPU chips? Pssh. We already knew workload-specific silicon was here. But two scale-up networking topologies? That's the actual Google TPU news. Workload-specific interconnects. Think about that. New Semi Doped with Vikram Sekar and Austin Lyons. Copper? Yep. Optics? Yep. What we cover: - TPU splits in two: 8t training, 8i inference. - Virgo: 47 Pb/s scale-out fabric, 100% OCS. - Boardfly scale-up: copper PCB + AECs inside racks, OCS between groups. 16 hops → 7. - Training uses 3D torus (Rubik's Cube). - Inference doesn't. Workload-specific topologies now. - Dragonfly traces to a 2008 paper by Kim, Dally, Scott, Abts. Abts went on to build Groq's interconnect before Nvidia. Chapters: 0:00 Intro 0:21 Two TPUs for two workloads 2:31 HBM, SRAM, and Axion CPUs 7:22 Why networking is the new bottleneck 17:14 Virgo: rebuilding scale-out on optics 25:24 3D torus Rubik's Cube scale-up for training 34:50 Boardfly: scale-up for MoE inference 42:07 Workload-specific everything $GOOGL

A masterclass on Google's TPU v8 Networking. Two TPU chips? Pssh. We already knew workload-specific silicon was here. But two scale-up networking topologies? That's the actual Google TPU news. Workload-specific interconnects. Think about that. New Semi Doped with Vikram Sekar and Austin Lyons. Copper? Yep. Optics? Yep. What we cover: - TPU splits in two: 8t training, 8i inference. - Virgo: 47 Pb/s scale-out fabric, 100% OCS. - Boardfly scale-up: copper PCB + AECs inside racks, OCS between groups. 16 hops → 7. - Training uses 3D torus (Rubik's Cube). - Inference doesn't. Workload-specific topologies now. - Dragonfly traces to a 2008 paper by Kim, Dally, Scott, Abts. Abts went on to build Groq's interconnect before Nvidia. Chapters: 0:00 Intro 0:21 Two TPUs for two workloads 2:31 HBM, SRAM, and Axion CPUs 7:22 Why networking is the new bottleneck 17:14 Virgo: rebuilding scale-out on optics 25:24 3D torus Rubik's Cube scale-up for training 34:50 Boardfly: scale-up for MoE inference 42:07 Workload-specific everything $GOOGL

Semi Doped

92,064 views • 4 months ago

DiffusionKit now supports Stable Diffusion 3 Medium MLX Python and Core ML Swift Inference work great for on-device inference on Mac! MLX: Core ML: Mac App: Hugging Face Diffusers App (Pending App Store review)

DiffusionKit now supports Stable Diffusion 3 Medium MLX Python and Core ML Swift Inference work great for on-device inference on Mac! MLX: Core ML: Mac App: Hugging Face Diffusers App (Pending App Store review)

argmax

74,009 views • 2 years ago

Introducing Real-time Transcription with Nvidia Parakeet - Same top accuracy as file transcription - Best-in-market 160 ms lips-to-screen latency - 744x more cost-efficient compared to cloud APIs - Available in Argmax Pro SDK starting today! Link in comments

Introducing Real-time Transcription with Nvidia Parakeet - Same top accuracy as file transcription - Best-in-market 160 ms lips-to-screen latency - 744x more cost-efficient compared to cloud APIs - Available in Argmax Pro SDK starting today! Link in comments

argmax

60,779 views • 1 year ago

Project Genesis is the closest thing we’ve seen to the Manhattan Project for science since 1939, what will this mean for America's AI dominance? - AWS planning $50B in U.S. government AI infrastructure + a 2.2 GW Indiana megacenter. - Google’s Ironwood TPU lands 4× performance and million-token multimodal context. - Amazon deploying 500,000 Tranium 2 chips for Anthropic-scale workloads. - Google is now rivaling NVIDIA by letting companies like Meta train frontier models directly on Google Cloud instead of buying GPUs.

Project Genesis is the closest thing we’ve seen to the Manhattan Project for science since 1939, what will this mean for America's AI dominance? - AWS planning $50B in U.S. government AI infrastructure + a 2.2 GW Indiana megacenter. - Google’s Ironwood TPU lands 4× performance and million-token multimodal context. - Amazon deploying 500,000 Tranium 2 chips for Anthropic-scale workloads. - Google is now rivaling NVIDIA by letting companies like Meta train frontier models directly on Google Cloud instead of buying GPUs.

Peter H. Diamandis, MD

41,936 views • 9 months ago

Windows on Arm is finally getting ready for gaming! Fortnite now runs on Snapdragon-powered Windows PCs. It's one of the major changes being rolled out to Windows on Arm PCs. Recently, Microsoft also enabled new CPU features for Windows on Arm emulation tech called 'Prism.' It now exposes AVX/AVX2, which allows high-performance computing. This enables support for game engines and resource-intensive apps that expect AVX and won’t start without it. You’ll see big gains in compatibility and mixed results in speed. Unlike traditional PCs, Snapdragon PCs offer up to 22 hours of battery life with no drop in performance when unplugged.

Windows on Arm is finally getting ready for gaming! Fortnite now runs on Snapdragon-powered Windows PCs. It's one of the major changes being rolled out to Windows on Arm PCs. Recently, Microsoft also enabled new CPU features for Windows on Arm emulation tech called 'Prism.' It now exposes AVX/AVX2, which allows high-performance computing. This enables support for game engines and resource-intensive apps that expect AVX and won’t start without it. You’ll see big gains in compatibility and mixed results in speed. Unlike traditional PCs, Snapdragon PCs offer up to 22 hours of battery life with no drop in performance when unplugged.

Windows Latest

80,587 views • 10 months ago

Introducing Argmax Local Server Run our state-of-the-art real-time transcription server directly on Mac! 0:31 Feature complete for AI Meeting Notes apps 0:49 Migrate from cloud APIs with 1 line of code 1:05 Fastest speech models with top accuracy 1:31 Other apps do not slow down Available starting today with Python, JavaScript, and Rust clients! Details in comments

Introducing Argmax Local Server Run our state-of-the-art real-time transcription server directly on Mac! 0:31 Feature complete for AI Meeting Notes apps 0:49 Migrate from cloud APIs with 1 line of code 1:05 Fastest speech models with top accuracy 1:31 Other apps do not slow down Available starting today with Python, JavaScript, and Rust clients! Details in comments

argmax

13,003 views • 1 year ago

Google Ironwood TPU Memory Hierarchy in 9 levels by hand ✍️ 1. Bit – The most basic unit of information, the on–off decision from which every number, tensor, and model state is ultimately constructed. 2. FP8 (1×8 → 8 bits) – Eight bits are grouped to form a floating-point value, typically used for inference, where reduced precision is a deliberate trade-off to maximize throughput and efficiency. 3. BF16 (×2 → 16 bits) – Two FP8-scale chunks are combined to gain more dynamic range and stability, while still staying friendly to high-throughput hardware. 4. Tensor tile (×1024 → 1K) – Data moves through the chip in blocks of 1024 values at a time, defining the granularity at which tensors are fetched and manipulated. 5. Matrix Multiplication Unit (MXU) (×64 → 64K) – A systolic array where matrix multiplication is not abstract but physical, with tensor tiles flowing through fixed hardware to achieve the highest possible throughput. 6. Vector Memory (VMEM) (×2048 → 128M) – On-chip working memory that holds activations, partial results, and intermediates, sized specifically to keep the systolic array busy without stalling. 7. Common Memory (CMEM) (×8 → 1 GB) – A small but critical shared memory sitting between VMEM and HBM, used for staging, accumulation, synchronization, and cross-lane coordination. 8. HBM (×96 → 96 GB) – Off-chip high-bandwidth memory where model weights and large states live, implemented as HBM3e with 16 stacks at 6 GB each, for a total of 96 GB. 9. Dual-Die (x2 → 192GB) – Two tightly coupled compute dies operate as a single logical accelerator, each with its own local HBM, effectively doubling memory capacity and bandwidth while allowing tensors and activations to stream seamlessly across dies as if they lived on one chip. I created this drawing for this week's seminar. I’ll take you through these 9 levels in a beginner-friendly way by hand ✍️. RSVP 👉

Google Ironwood TPU Memory Hierarchy in 9 levels by hand ✍️ 1. Bit – The most basic unit of information, the on–off decision from which every number, tensor, and model state is ultimately constructed. 2. FP8 (1×8 → 8 bits) – Eight bits are grouped to form a floating-point value, typically used for inference, where reduced precision is a deliberate trade-off to maximize throughput and efficiency. 3. BF16 (×2 → 16 bits) – Two FP8-scale chunks are combined to gain more dynamic range and stability, while still staying friendly to high-throughput hardware. 4. Tensor tile (×1024 → 1K) – Data moves through the chip in blocks of 1024 values at a time, defining the granularity at which tensors are fetched and manipulated. 5. Matrix Multiplication Unit (MXU) (×64 → 64K) – A systolic array where matrix multiplication is not abstract but physical, with tensor tiles flowing through fixed hardware to achieve the highest possible throughput. 6. Vector Memory (VMEM) (×2048 → 128M) – On-chip working memory that holds activations, partial results, and intermediates, sized specifically to keep the systolic array busy without stalling. 7. Common Memory (CMEM) (×8 → 1 GB) – A small but critical shared memory sitting between VMEM and HBM, used for staging, accumulation, synchronization, and cross-lane coordination. 8. HBM (×96 → 96 GB) – Off-chip high-bandwidth memory where model weights and large states live, implemented as HBM3e with 16 stacks at 6 GB each, for a total of 96 GB. 9. Dual-Die (x2 → 192GB) – Two tightly coupled compute dies operate as a single logical accelerator, each with its own local HBM, effectively doubling memory capacity and bandwidth while allowing tensors and activations to stream seamlessly across dies as if they lived on one chip. I created this drawing for this week's seminar. I’ll take you through these 9 levels in a beginner-friendly way by hand ✍️. RSVP 👉

Tom Yeh

30,489 views • 7 months ago

Introducing WhisperKit

Introducing WhisperKit

argmax

98,413 views • 2 years ago

On this day in 2009, Kelly Clarkson went to No.1 on the UK singles chart with “My Life Would Suck Without You”. The US singer, who w💿n the first season of American Idol in 2002, became the only American Idol contestant to ever top the UK charts.

On this day in 2009, Kelly Clarkson went to No.1 on the UK singles chart with “My Life Would Suck Without You”. The US singer, who w💿n the first season of American Idol in 2002, became the only American Idol contestant to ever top the UK charts.

💿n This Day in Music

19,781 views • 6 months ago

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

QVAC

108,835 views • 5 months ago

The most retarded thing about Zack's Pixel Fold battery explosion is his claim (and people repeating it) that this is the first time ever that this happened. Well YEAH because it was also the FIRST TIME that he pushed it way beyond his usual destruction and kept trying to fold it over at the breaking point (the antenna) way after it already got destroyed On all his other ""tests"" he always stops when the phone breaks. Not to much here. After it broke he tried to completely fold it and THAT was the point that damaged the battery which resulted in it starting to smoke. Don't blame fucking Google, blame his retarded ""testing"" It's literally the FIRST time he did this to a phone. Not the first time this happening in his tests". It would happen almost every single time if he did this to all his phones and foldables. For comparison on the Fold 7 he just stopped after a quick bent. On the Pixel he didn't.

The most retarded thing about Zack's Pixel Fold battery explosion is his claim (and people repeating it) that this is the first time ever that this happened. Well YEAH because it was also the FIRST TIME that he pushed it way beyond his usual destruction and kept trying to fold it over at the breaking point (the antenna) way after it already got destroyed On all his other ""tests"" he always stops when the phone breaks. Not to much here. After it broke he tried to completely fold it and THAT was the point that damaged the battery which resulted in it starting to smoke. Don't blame fucking Google, blame his retarded ""testing"" It's literally the FIRST time he did this to a phone. Not the first time this happening in his tests". It would happen almost every single time if he did this to all his phones and foldables. For comparison on the Fold 7 he just stopped after a quick bent. On the Pixel he didn't.

Tim 🏳️‍🌈

59,445 views • 10 months ago

🚨 do you understand what happened to Apple.. At WWDC, Apple confirmed that Siri's cloud layer now runs on a custom Google Gemini model with 1.2 trillion parameters, licensed for roughly $1 billion a year. Strategically, this is a reversal. The company that built its brand on privacy and vertical integration has handed the core of its AI stack to a direct competitor. The same capabilities Apple announced at WWDC 2024 and never shipped — personal context, on-screen awareness, cross-app actions — now depend on Google Cloud. → $250M settlement in May for marketing Siri features that didn't exist → debuted at WWDC 2024, delayed March 2025, ads pulled → Tim Cook's final keynote before stepping down September 1 → Siri's cloud inference now runs on Google's infrastructure Apple didn't close the AI gap. It leased it. Pragmatic move or structural capitulation in the AI race?

Alex Stone

1,270,500 views • 2 months ago

Destra Edge — $DSYNC From Idea to Reality Yesterday, we introduced Destra Edge. Today, we show you what it means. For too long, AI has lived in data centers. Expensive. Centralized. Controlled by a few. Meanwhile, billions of smartphones sit idle. Powerful GPUs. Doing nothing. That doesn’t make sense. So we’re building something different. Destra Edge turns everyday smartphones into a global AI inference network. No servers. No cloud monopolies. Just real compute at the edge. This video shows the system taking shape — how inference is distributed, how it runs on-device, and how results come together without central control. This is not finished. And that’s the point. You’re watching it being built. Layer by layer. In public. The magic is in the scale. More phones means more power. More power means stronger AI. The network grows simply because people join. No passive mining. No fake activity. Only real GPU inference. Measured. Verified. Rewarded. AI shouldn’t live in warehouses. It should live in your pocket. This is edge-level decentralization. This is permissionless inference. This is Destra Network.

Destra Edge — $DSYNC From Idea to Reality Yesterday, we introduced Destra Edge. Today, we show you what it means. For too long, AI has lived in data centers. Expensive. Centralized. Controlled by a few. Meanwhile, billions of smartphones sit idle. Powerful GPUs. Doing nothing. That doesn’t make sense. So we’re building something different. Destra Edge turns everyday smartphones into a global AI inference network. No servers. No cloud monopolies. Just real compute at the edge. This video shows the system taking shape — how inference is distributed, how it runs on-device, and how results come together without central control. This is not finished. And that’s the point. You’re watching it being built. Layer by layer. In public. The magic is in the scale. More phones means more power. More power means stronger AI. The network grows simply because people join. No passive mining. No fake activity. Only real GPU inference. Measured. Verified. Rewarded. AI shouldn’t live in warehouses. It should live in your pocket. This is edge-level decentralization. This is permissionless inference. This is Destra Network.

Destra Network

20,283 views • 8 months ago

Demis Hassabis said that 90% of the breakthroughs the entire AI industry runs on came out of Google. Every competitor is building on work his teams did first. He said it without flinching. Over the last decade, he claims 90% or more of the big breakthroughs that underpin modern AI came from Google Brain or DeepMind. Transformers. AlphaGo. The reinforcement learning everything now depends on. Transformers are the T in GPT. The architecture that made every large language model possible came out of Google in 2017, in a paper called Attention Is All You Need. His point was simple. Every rival lab is running on foundations his people poured. But here is what he left out. Of the eight authors who wrote that transformer paper, all eight left Google. They went on to found or join OpenAI, Anthropic, and their own startups now worth billions. Google invented the engine. Then watched the inventors walk out the door and build the competition. He says Google still has the biggest research bench of any lab on earth, and the talent war is the most ferocious tech has ever seen. Everyone treats this race like the labs are evenly matched. The man who started fifteen years early does not see it that way. The people who left with his blueprint might be the reason he has to keep proving it. (Watch the full interview at Semafor on YouTube)

Demis Hassabis said that 90% of the breakthroughs the entire AI industry runs on came out of Google. Every competitor is building on work his teams did first. He said it without flinching. Over the last decade, he claims 90% or more of the big breakthroughs that underpin modern AI came from Google Brain or DeepMind. Transformers. AlphaGo. The reinforcement learning everything now depends on. Transformers are the T in GPT. The architecture that made every large language model possible came out of Google in 2017, in a paper called Attention Is All You Need. His point was simple. Every rival lab is running on foundations his people poured. But here is what he left out. Of the eight authors who wrote that transformer paper, all eight left Google. They went on to found or join OpenAI, Anthropic, and their own startups now worth billions. Google invented the engine. Then watched the inventors walk out the door and build the competition. He says Google still has the biggest research bench of any lab on earth, and the talent war is the most ferocious tech has ever seen. Everyone treats this race like the labs are evenly matched. The man who started fifteen years early does not see it that way. The people who left with his blueprint might be the reason he has to keep proving it. (Watch the full interview at Semafor on YouTube)

Ihtesham Ali

35,961 views • 2 months ago

On Friday, I hosted a Space with Jonathan Ross, the founder and CEO of Groq Inc - a company I invested in that is building custom chips for AI inference. Jonathan, a former high-school dropout, entered the chip industry while working on ad optimization at Google’s New York office. Jonathan overheard the speech recognition team complaining that they couldn't get enough compute. These were the early days of AI, and machine learning wasn’t really a thing yet. So he asked for some budget from Google and started putting together a chip-based machine learning accelerator for them. During the day, Jonathan would work in the normal ads part of the business, and at night, he would work with the accelerator team. After winning approval from Google, Jonathan and his team built a new chip called the Tensor Processing Unit, and began deploying it across Google’s data centers within a year. The TPU was a huge success within Google, eventually underpinning more than 50% of all of Google’s compute power. When the other hyper-scalers learned of this success, they tried to hire Jonathan to build custom chips for them too. During this process, it became increasingly clear to Jonathan that a gap would emerge between companies that had access to next-gen compute and companies that didn’t. So he founded Groq and set out to build a chip that would be available to everyone. I led Groq’s founding investment in 2016, and since then, Jonathan and his team have developed several types of AI hardware including the Language Processing Unit (LPU), a new type of silicon that is hyper-efficient at running inference for LLMs. In our conversation on Friday, we discussed the founding story of Groq, what you need for great AI hardware, large language models, and some of the implications for the key players in AI. It’s one of the most interesting conversations I’ve had on AI with a lot of learnings. You can listen to our conversation below:

On Friday, I hosted a Space with Jonathan Ross, the founder and CEO of Groq Inc - a company I invested in that is building custom chips for AI inference. Jonathan, a former high-school dropout, entered the chip industry while working on ad optimization at Google’s New York office. Jonathan overheard the speech recognition team complaining that they couldn't get enough compute. These were the early days of AI, and machine learning wasn’t really a thing yet. So he asked for some budget from Google and started putting together a chip-based machine learning accelerator for them. During the day, Jonathan would work in the normal ads part of the business, and at night, he would work with the accelerator team. After winning approval from Google, Jonathan and his team built a new chip called the Tensor Processing Unit, and began deploying it across Google’s data centers within a year. The TPU was a huge success within Google, eventually underpinning more than 50% of all of Google’s compute power. When the other hyper-scalers learned of this success, they tried to hire Jonathan to build custom chips for them too. During this process, it became increasingly clear to Jonathan that a gap would emerge between companies that had access to next-gen compute and companies that didn’t. So he founded Groq and set out to build a chip that would be available to everyone. I led Groq’s founding investment in 2016, and since then, Jonathan and his team have developed several types of AI hardware including the Language Processing Unit (LPU), a new type of silicon that is hyper-efficient at running inference for LLMs. In our conversation on Friday, we discussed the founding story of Groq, what you need for great AI hardware, large language models, and some of the implications for the key players in AI. It’s one of the most interesting conversations I’ve had on AI with a lot of learnings. You can listen to our conversation below:

Chamath Palihapitiya

326,752 views • 2 years ago

The world of tomorrow cannot run on a rented cloud. 🚫 With 10 billion humans and 10 billion autonomous agents, intelligence must be embedded at the edge - not centralized in a server farm. The QVAC SDK is the invisible engine for this transition. We’ve built the foundational toolkit for the next era: highly efficient, fully modular, and 100% sovereign. From a single light to an industrial grid, the power to build local-first AI is now in your hands. The revolution will not be hosted. It will be local. Learn more:

The world of tomorrow cannot run on a rented cloud. 🚫 With 10 billion humans and 10 billion autonomous agents, intelligence must be embedded at the edge - not centralized in a server farm. The QVAC SDK is the invisible engine for this transition. We’ve built the foundational toolkit for the next era: highly efficient, fully modular, and 100% sovereign. From a single light to an industrial grid, the power to build local-first AI is now in your hands. The revolution will not be hosted. It will be local. Learn more:

QVAC

2,909,096 views • 3 months ago

Today we announced our new Fairwater datacenter in Atlanta, connected with our first Fairwater site in Wisconsin and our broader Azure footprint to create the world’s first AI superfactory. Fairwater exemplifies our vision for a fungible fleet: infra that can serve any workload, anywhere, on fit-for-purpose accelerators and network paths, with maximum performance and efficiency. AI workloads have evolved beyond large-scale pre-training. Today, they encompass fine-tuning, reinforcement learning (RL), synthetic data generation, evaluation pipelines, and more. Fairwater is built to support this full lifecycle: Max density: Fairwater’s two-story design and liquid cooling system lets us place racks in three dimensions and pack them with GPUs as densely as possible, minimizing cable runs and improving latency and effective bandwidth. Fleet: Each Fairwater DC can integrate hundreds of thousands of the latest NVIDIA GPUs into a single coherent cluster. This provides flexible infra that can support the full spectrum of workloads, and ensure no GPU is left unnecessarily idle. And that’s on top of the more than 100,000 GB300s coming online this quarter alone for inference across the rest of our fleet. For us, it’s all about turning every gigawatt into the maximum number of useful tokens. Not every GW is created equal! Planet-scale: Every Fairwater DC will connect through our continent-spanning AI WAN to prior generations of AI supercomputers, forming a truly fungible pool of compute. This enables developers to scale beyond the capacity of a single site and dynamically land workloads on the right infra for their needs. Together, these innovations let us bring together different generations of silicon and AI systems across DCs and geos into a single elastic system that scales seamlessly across training and inference workloads And this elastic AI capacity is all available alongside all the other cloud services (compute, storage, databases, app services) that AI agents and workloads need. This is what we mean when we talk about building a fungible fleet – a single, unified platform that pushes the limits of performance per watt and per dollar. Read more:

Today we announced our new Fairwater datacenter in Atlanta, connected with our first Fairwater site in Wisconsin and our broader Azure footprint to create the world’s first AI superfactory. Fairwater exemplifies our vision for a fungible fleet: infra that can serve any workload, anywhere, on fit-for-purpose accelerators and network paths, with maximum performance and efficiency. AI workloads have evolved beyond large-scale pre-training. Today, they encompass fine-tuning, reinforcement learning (RL), synthetic data generation, evaluation pipelines, and more. Fairwater is built to support this full lifecycle: Max density: Fairwater’s two-story design and liquid cooling system lets us place racks in three dimensions and pack them with GPUs as densely as possible, minimizing cable runs and improving latency and effective bandwidth. Fleet: Each Fairwater DC can integrate hundreds of thousands of the latest NVIDIA GPUs into a single coherent cluster. This provides flexible infra that can support the full spectrum of workloads, and ensure no GPU is left unnecessarily idle. And that’s on top of the more than 100,000 GB300s coming online this quarter alone for inference across the rest of our fleet. For us, it’s all about turning every gigawatt into the maximum number of useful tokens. Not every GW is created equal! Planet-scale: Every Fairwater DC will connect through our continent-spanning AI WAN to prior generations of AI supercomputers, forming a truly fungible pool of compute. This enables developers to scale beyond the capacity of a single site and dynamically land workloads on the right infra for their needs. Together, these innovations let us bring together different generations of silicon and AI systems across DCs and geos into a single elastic system that scales seamlessly across training and inference workloads And this elastic AI capacity is all available alongside all the other cloud services (compute, storage, databases, app services) that AI agents and workloads need. This is what we mean when we talk about building a fungible fleet – a single, unified platform that pushes the limits of performance per watt and per dollar. Read more:

Satya Nadella

908,065 views • 9 months ago