Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

A masterclass on Google's TPU v8 Networking. Two TPU chips? Pssh. We already knew workload-specific silicon was here. But two scale-up networking topologies? That's the actual Google TPU news. Workload-specific interconnects. Think about that. New Semi Doped with Vikram Sekar and Austin Lyons. Copper? Yep. Optics? Yep. What we... cover: - TPU splits in two: 8t training, 8i inference. - Virgo: 47 Pb/s scale-out fabric, 100% OCS. - Boardfly scale-up: copper PCB + AECs inside racks, OCS between groups. 16 hops → 7. - Training uses 3D torus (Rubik's Cube). - Inference doesn't. Workload-specific topologies now. - Dragonfly traces to a 2008 paper by Kim, Dally, Scott, Abts. Abts went on to build Groq's interconnect before Nvidia. Chapters: 0:00 Intro 0:21 Two TPUs for two workloads 2:31 HBM, SRAM, and Axion CPUs 7:22 Why networking is the new bottleneck 17:14 Virgo: rebuilding scale-out on optics 25:24 3D torus Rubik's Cube scale-up for training 34:50 Boardfly: scale-up for MoE inference 42:07 Workload-specific everything $GOOGLshow more

Semi Doped

2,677 subscribers

92,064 görüntüleme • 2 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Argmax now runs on Google Tensor TPU, the first-ever SDK to harness this edge inference accelerator! Tensor TPU enabled us to deploy billion-scale transformers reliably on Pixel phones without impacting battery life or resource contention with traditional workloads.

Argmax now runs on Google Tensor TPU, the first-ever SDK to harness this edge inference accelerator! Tensor TPU enabled us to deploy billion-scale transformers reliably on Pixel phones without impacting battery life or resource contention with traditional workloads.

argmax

26,957 görüntüleme • 2 ay önce

The optical networking supercycle is here! In this podcast, and Vikram Sekar go through all the tech jargon and explain what everything means. In just about 45 mins, you will know everything required to keep up with the next revolution in AI. Chapters 00:00 Introduction to AI and CPU Bottlenecks 03:00 The Rise of Silicon Photonics 06:01 Understanding Optical Networking and Data Centers 08:49 Scale Across: Connecting Data Centers 11:56 Scale Out: Optimizing Data Center Connectivity 14:53 Scale Up: The Future of GPU Connectivity 23:32 The Shift from Copper to Optical Connections 26:13 Challenges and Reliability of Lasers 30:47 Understanding Co-Packaged Optics 34:17 Market Dynamics: Demand and Supply of Lasers 40:46 Emerging Technologies: Optical Circuit Switches

The optical networking supercycle is here! In this podcast, and Vikram Sekar go through all the tech jargon and explain what everything means. In just about 45 mins, you will know everything required to keep up with the next revolution in AI. Chapters 00:00 Introduction to AI and CPU Bottlenecks 03:00 The Rise of Silicon Photonics 06:01 Understanding Optical Networking and Data Centers 08:49 Scale Across: Connecting Data Centers 11:56 Scale Out: Optimizing Data Center Connectivity 14:53 Scale Up: The Future of GPU Connectivity 23:32 The Shift from Copper to Optical Connections 26:13 Challenges and Reliability of Lasers 30:47 Understanding Co-Packaged Optics 34:17 Market Dynamics: Demand and Supply of Lasers 40:46 Emerging Technologies: Optical Circuit Switches

Semi Doped

21,842 görüntüleme • 4 ay önce

Every AI lab is starving for compute. Except Google. I spoke with Thomas Kurian, Google Cloud's CEO, to understand why Google doesn't just hoard compute before AGI, their relationship with Anthropic, and that viral tweet about Google's engineering culture. Watch now: 0:00 – Intro 0:42 – Google's Insane Compute Capacity 03:17 – TPU Monetization 05:24 – Why Google Doesn't Hoard Compute? 08:02 – Datacenter Buildout 15:01 - Does AGI Mean Job Displacement? 17:55 - NVIDIA vs TPU (Total Cost of Ownership) 23:25 - 8th Gen TPU 24:32 - Training vs. Inference 30:53 - Google's "Extreme Co-design" 35:01 - Working with Anthropic 37:46 - Serving Mythos-sized Models (10T) 41:42 - Google engineering culture (Steve Yagge tweet) 48:27 - Cybersecurity 51:50 - What keeps Thomas up at night?

Every AI lab is starving for compute. Except Google. I spoke with Thomas Kurian, Google Cloud's CEO, to understand why Google doesn't just hoard compute before AGI, their relationship with Anthropic, and that viral tweet about Google's engineering culture. Watch now: 0:00 – Intro 0:42 – Google's Insane Compute Capacity 03:17 – TPU Monetization 05:24 – Why Google Doesn't Hoard Compute? 08:02 – Datacenter Buildout 15:01 - Does AGI Mean Job Displacement? 17:55 - NVIDIA vs TPU (Total Cost of Ownership) 23:25 - 8th Gen TPU 24:32 - Training vs. Inference 30:53 - Google's "Extreme Co-design" 35:01 - Working with Anthropic 37:46 - Serving Mythos-sized Models (10T) 41:42 - Google engineering culture (Steve Yagge tweet) 48:27 - Cybersecurity 51:50 - What keeps Thomas up at night?

Matthew Berman

505,546 görüntüleme • 2 ay önce

Our approach to AI infra is simple: build the most fungible and flexible fleet to meet the real world's needs across inference and training as Scott Guthrie shared with Alex Kantrowitz. And we are already doing it at scale today, as we power the biggest AI workloads like Copilot and ChatGPT, APIs that power 3P products & enterprise workloads and high scale training.

Our approach to AI infra is simple: build the most fungible and flexible fleet to meet the real world's needs across inference and training as Scott Guthrie shared with Alex Kantrowitz. And we are already doing it at scale today, as we power the biggest AI workloads like Copilot and ChatGPT, APIs that power 3P products & enterprise workloads and high scale training.

Satya Nadella

114,478 görüntüleme • 9 ay önce

#generativeAI models have grown 10x in parameters per year for the last five years 🤯 That's why we're announcing Cloud TPU v5p (in preview)—the most powerful, scalable, and flexible TPU to date. P.S. Gemini was trained at scale on Cloud TPUs, too! #GeminiAI

#generativeAI models have grown 10x in parameters per year for the last five years 🤯 That's why we're announcing Cloud TPU v5p (in preview)—the most powerful, scalable, and flexible TPU to date. P.S. Gemini was trained at scale on Cloud TPUs, too! #GeminiAI

Google Cloud Tech

142,604 görüntüleme • 2 yıl önce

NVIDIA HAS TAPED OUT 6 RUBIN CHIPS AT TSMC - New CPU (Vera - 88 ARM cores with SMT) - New GPU variants (R100 with HBM4) - Scale-up NVLink switch - Silicon photonics processor - Networking silicon - Complete chipset overhaul Reminder that Rubin is on-schedule for mass production in the 2nd half of 2026

NVIDIA HAS TAPED OUT 6 RUBIN CHIPS AT TSMC - New CPU (Vera - 88 ARM cores with SMT) - New GPU variants (R100 with HBM4) - Scale-up NVLink switch - Silicon photonics processor - Networking silicon - Complete chipset overhaul Reminder that Rubin is on-schedule for mass production in the 2nd half of 2026

NIK

78,410 görüntüleme • 10 ay önce

Whiplash week. Optics? Copper? Both? Mon AM: Nvidia bets $4B on optics. Mon PM: Credo posts 200% YoY growth on copper. Wed PM: Hock Tan claims 400G/lane works over copper, potentially pushing CPO past 2030. 48 hours of whiplash. Optics? Copper? The answer is both. The question is when. This week and Vikram Sekar unpack: - Nvidia locking up laser supply - Credo’s blowout quarter and the reliability thesis - Broadcom’s copper bombshell - A 4D chess theory on why Hock Tan downplays optics when Broadcom is a CPO company Chapters (00:00) - Newsletter Plugs: Groq LPUs & Broadcom’s Laser Business (03:15) - Dynamo & the Rise of Workload-Specific Hardware (08:04) - Austin’s Broadcom Laser Deep Dive (09:53) - The Week’s Whiplash: Optics Monday, Copper Wednesday (17:50) - Why Nvidia Invested $4B: Geopolitics, Supply & the HBM Playbook (24:15) - CPO Lasers & Optical Circuit Switches (26:16) - Credo Earnings: 200% YoY Growth & the Copper Bull Case (31:09) - Reliability, AECs & Oracle’s GPU Cluster Problem (35:48) - Credo’s Optics Play: Micro-LED Active Cables & the CPO Timing Risk (38:45) - Broadcom Earnings: Hock Tan’s Copper Bombshell (43:34) - Customer-Owned Tooling: Hock Tan Says “Good Luck” (44:25) - Vik’s 4D Chess Theory: Why Hock Tan Talks Up Copper (47:03) - Wrap-Up: It’s Both — The Real Question Is Timing $AVGO $CRDO $NVDA

Whiplash week. Optics? Copper? Both? Mon AM: Nvidia bets $4B on optics. Mon PM: Credo posts 200% YoY growth on copper. Wed PM: Hock Tan claims 400G/lane works over copper, potentially pushing CPO past 2030. 48 hours of whiplash. Optics? Copper? The answer is both. The question is when. This week and Vikram Sekar unpack: - Nvidia locking up laser supply - Credo’s blowout quarter and the reliability thesis - Broadcom’s copper bombshell - A 4D chess theory on why Hock Tan downplays optics when Broadcom is a CPO company Chapters (00:00) - Newsletter Plugs: Groq LPUs & Broadcom’s Laser Business (03:15) - Dynamo & the Rise of Workload-Specific Hardware (08:04) - Austin’s Broadcom Laser Deep Dive (09:53) - The Week’s Whiplash: Optics Monday, Copper Wednesday (17:50) - Why Nvidia Invested $4B: Geopolitics, Supply & the HBM Playbook (24:15) - CPO Lasers & Optical Circuit Switches (26:16) - Credo Earnings: 200% YoY Growth & the Copper Bull Case (31:09) - Reliability, AECs & Oracle’s GPU Cluster Problem (35:48) - Credo’s Optics Play: Micro-LED Active Cables & the CPO Timing Risk (38:45) - Broadcom Earnings: Hock Tan’s Copper Bombshell (43:34) - Customer-Owned Tooling: Hock Tan Says “Good Luck” (44:25) - Vik’s 4D Chess Theory: Why Hock Tan Talks Up Copper (47:03) - Wrap-Up: It’s Both — The Real Question Is Timing $AVGO $CRDO $NVDA

Semi Doped

33,695 görüntüleme • 4 ay önce

$AVGO helps hyperscalers build the custom chips and networking fabric needed to run massive AI systems. If $GOOGL, $META, OpenAI or Anthropic want to build their own inference chips then Broadcom is one of the first calls because it has the IP, packaging expertise and networking portfolio required to scale custom AI systems. Thats why Broadcom remains one of the strongest infrastructure positions in the AI economy.

$AVGO helps hyperscalers build the custom chips and networking fabric needed to run massive AI systems. If $GOOGL, $META, OpenAI or Anthropic want to build their own inference chips then Broadcom is one of the first calls because it has the IP, packaging expertise and networking portfolio required to scale custom AI systems. Thats why Broadcom remains one of the strongest infrastructure positions in the AI economy.

Shay Boloor

77,765 görüntüleme • 1 ay önce

Biggest scale i could squeeze on the Prusa3D XL! Fake feather in PETG and foam TPU. Free STL on my website

Biggest scale i could squeeze on the Prusa3D XL! Fake feather in PETG and foam TPU. Free STL on my website

Willow Creative

14,675 görüntüleme • 1 yıl önce

We are excited to share a preview of our peer-to-peer decentralized inference stack Engineered for consumer GPUs and high-latency networks — plus a research roadmap to scale it to a planetary-scale decentralized inference engine.

We are excited to share a preview of our peer-to-peer decentralized inference stack Engineered for consumer GPUs and high-latency networks — plus a research roadmap to scale it to a planetary-scale decentralized inference engine.

Prime Intellect

103,625 görüntüleme • 1 yıl önce

We need to scale up the Kardashev scale ASAP We need to massively scale energy production and look at new algorithms/forms of compute for greater utilization/efficiency with our current grid

We need to scale up the Kardashev scale ASAP We need to massively scale energy production and look at new algorithms/forms of compute for greater utilization/efficiency with our current grid

Beff – e/acc

50,953 görüntüleme • 1 yıl önce

those little details™ Scale up and down on scroll with CSS 📜✨ a { animation: scale, scale; animation-direction: normal, reverse; animation-timeline: view(inline); animation-range: entry, exit; } @keyframes scale { 0% { scale: 0; } }

those little details™ Scale up and down on scroll with CSS 📜✨ a { animation: scale, scale; animation-direction: normal, reverse; animation-timeline: view(inline); animation-range: entry, exit; } @keyframes scale { 0% { scale: 0; } }

jhey ʕ•ᴥ•ʔ

246,038 görüntüleme • 2 yıl önce

🚨NEW: Elon Musk on his DOGE workload: "It is quite a challenge. Although I've interfaced with government tremendously, this is my first time actually trying to improve government. So there will be some initial training workload that is significant, and then we'll move to inference. which is several orders of magnitude easier than training."

🚨NEW: Elon Musk on his DOGE workload: "It is quite a challenge. Although I've interfaced with government tremendously, this is my first time actually trying to improve government. So there will be some initial training workload that is significant, and then we'll move to inference. which is several orders of magnitude easier than training."

DogeDesigner

1,108,767 görüntüleme • 1 yıl önce

$AMD CEO Lisa Su said Helios has industry-leading specs and is on track to launch this year. Built with $META, Helios targets large-scale AI training & inference aiming for up to 10x performance where performance per dollar matters.

$AMD CEO Lisa Su said Helios has industry-leading specs and is on track to launch this year. Built with $META, Helios targets large-scale AI training & inference aiming for up to 10x performance where performance per dollar matters.

Shay Boloor

117,981 görüntüleme • 5 ay önce

NVIDIA Vera Rubin is in full production and arrives just in time for the next frontier of AI. The #NVIDIARubin platform uses extreme co-design across six new chips to accelerate agentic AI, advanced reasoning and massive-scale MoE model inference. NVIDIA Vera Rubin NVL72: Six new chips, one AI supercomputer — see how we built it.

NVIDIA Vera Rubin is in full production and arrives just in time for the next frontier of AI. The #NVIDIARubin platform uses extreme co-design across six new chips to accelerate agentic AI, advanced reasoning and massive-scale MoE model inference. NVIDIA Vera Rubin NVL72: Six new chips, one AI supercomputer — see how we built it.

NVIDIA Data Center

27,643 görüntüleme • 5 ay önce

Unbiased AI -- at Scale Truly Open, inference at scale, running on Subnet 4. Powered by the Bittensor ecosystem. Full episode on Youtube and Spotify:

Unbiased AI -- at Scale Truly Open, inference at scale, running on Subnet 4. Powered by the Bittensor ecosystem. Full episode on Youtube and Spotify:

Openτensor Foundaτion

45,393 görüntüleme • 1 yıl önce

ROCm 7 is here! ✅MI350 Series support ✅Enhanced usability and performance ✅Advanced AI features (distributed inference, large-scale training) ✅Easier enterprise AI deployment

ROCm 7 is here! ✅MI350 Series support ✅Enhanced usability and performance ✅Advanced AI features (distributed inference, large-scale training) ✅Easier enterprise AI deployment

AMD

24,099 görüntüleme • 11 ay önce

The first-ever hydrogen-powered NVIDIA GB300 NVL72 systems are online. Training & inference at scale — with zero emissions, zero water. Lambda × Supermicro × ECL →

The first-ever hydrogen-powered NVIDIA GB300 NVL72 systems are online. Training & inference at scale — with zero emissions, zero water. Lambda × Supermicro × ECL →

Lambda

17,276 görüntüleme • 9 ay önce

1/ $AMD is a 10x opportunity everybody ignores. Lisa Su believes that inference will be a bigger market than training. Training is the cost center of AI workload while inference is where you make money. $AMD has an advantage here. Let me explain why: 🧵

1/ $AMD is a 10x opportunity everybody ignores. Lisa Su believes that inference will be a bigger market than training. Training is the cost center of AI workload while inference is where you make money. $AMD has an advantage here. Let me explain why: 🧵

Oguz Erkan

413,014 görüntüleme • 1 yıl önce

Jensen Huang says $NVDA invested $2B in $MRVL because the TAM for networking and custom silicon is entering an inference boom. Nvidia is becoming the platform layer for a more distributed AI factory architecture increasingly built around its networking fabric.

Jensen Huang says $NVDA invested $2B in $MRVL because the TAM for networking and custom silicon is entering an inference boom. Nvidia is becoming the platform layer for a more distributed AI factory architecture increasingly built around its networking fabric.

Shay Boloor

111,005 görüntüleme • 3 ay önce