Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Inference Chips for Agent Workflows Diana Most AI chips are designed for "prompt in, response out." Agents don't work that way. They loop, branch, and hold context across dozens of steps, and current GPUs hit 30–40% utilization as a result. That gap is where purpose-built silicon wins.

Y Combinator

1,620,501 subscribers

707,760 views • 2 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Chips are the foundation of every AI experience. That's why understanding the hardware behind the software has never mattered more. When you ask a chatbot a question or generate an image, there's a physical chip somewhere doing the heavy lifting. For most of computing history, that was a CPU—the "brain" of a computer—great for general tasks needed to run software and operating systems. AI is more complex: for workloads called training and inference, AI needs to perform trillions of calculations in parallel. That's where AI accelerators come in. Purpose-built accelerators can deliver significantly better performance and efficiency than general-purpose chips. Amazon Web Services Trainium chips are an example—purpose-built for AI training and inference. New chips for a new era. ⬇️

Chips are the foundation of every AI experience. That's why understanding the hardware behind the software has never mattered more. When you ask a chatbot a question or generate an image, there's a physical chip somewhere doing the heavy lifting. For most of computing history, that was a CPU—the "brain" of a computer—great for general tasks needed to run software and operating systems. AI is more complex: for workloads called training and inference, AI needs to perform trillions of calculations in parallel. That's where AI accelerators come in. Purpose-built accelerators can deliver significantly better performance and efficiency than general-purpose chips. Amazon Web Services Trainium chips are an example—purpose-built for AI training and inference. New chips for a new era. ⬇️

Amazon

27,151 views • 1 month ago

Dojo 2, Dojo 3 Chips and AI5 Chips Are Critical for Tesla and Battle for Future of AI. Tesla and XAI are buying about one million AI GPUs this year. This would be about 20% of the entire Nvidia production of B200/B300 chips that will be made in 2025. If XAI were to buy a mix of half Dojo 2/3 chips in 2025 and 2026 this could make them bigger than AMD. AMD is making about 400,000 AI GPU chips. Tesla will switch to AI5 chips for driving its cars but those chips will also act as inference chips with about 500 Teraflops of compute. Tesla could make 3-4 million AI5 chips in 2026. This would make Tesla the largest maker and user of AI inference chips in the world. Tesla could become the number 2 maker of AI training chips just by supplying a lot of those chips to XAI and Tesla internal use. Tesla is and will be the number one maker and user of AI inference chips. Sawyer Merritt Elon Musk Randy Kirk Herbert Ong Warren Redlich - Chasing Dreams 🇺🇸 Ale𝕏andra Merz 🇺🇲 Ray Emmet Peppers Bradford Ferguson

Dojo 2, Dojo 3 Chips and AI5 Chips Are Critical for Tesla and Battle for Future of AI. Tesla and XAI are buying about one million AI GPUs this year. This would be about 20% of the entire Nvidia production of B200/B300 chips that will be made in 2025. If XAI were to buy a mix of half Dojo 2/3 chips in 2025 and 2026 this could make them bigger than AMD. AMD is making about 400,000 AI GPU chips. Tesla will switch to AI5 chips for driving its cars but those chips will also act as inference chips with about 500 Teraflops of compute. Tesla could make 3-4 million AI5 chips in 2026. This would make Tesla the largest maker and user of AI inference chips in the world. Tesla could become the number 2 maker of AI training chips just by supplying a lot of those chips to XAI and Tesla internal use. Tesla is and will be the number one maker and user of AI inference chips. Sawyer Merritt Elon Musk Randy Kirk Herbert Ong Warren Redlich - Chasing Dreams 🇺🇸 Ale𝕏andra Merz 🇺🇲 Ray Emmet Peppers Bradford Ferguson

nextbigfuture

23,809 views • 1 year ago

We are innovating the full-stack AI infrastructure platform on a one-year rhythm — pushing the boundaries of inference performance across chips, systems, and software. NVIDIA Blackwell is the most advanced AI platform ever built — continuously optimized for record-breaking performance. Groundbreaking advancements include the NVIDIA-designed NVFP4 format, enabling high inference performance and accuracy. #GTCParis at #VivaTech

We are innovating the full-stack AI infrastructure platform on a one-year rhythm — pushing the boundaries of inference performance across chips, systems, and software. NVIDIA Blackwell is the most advanced AI platform ever built — continuously optimized for record-breaking performance. Groundbreaking advancements include the NVIDIA-designed NVFP4 format, enabling high inference performance and accuracy. #GTCParis at #VivaTech

NVIDIA

80,625 views • 1 year ago

No wonder Jensen got so defensive over $NVDA GPUs against custom chips. $GOOG will start using different custom chips for inference and training. $AMZN has already adopted this approach as they have Trainium for training and Inferentia for inference. Hyperscalers obviously see better price/performance in this approach, and this is where the industry is headed. $AVGO and $MRVL are poised to be big winners, while $NVDA dominance will be under increasing pressure.

No wonder Jensen got so defensive over $NVDA GPUs against custom chips. $GOOG will start using different custom chips for inference and training. $AMZN has already adopted this approach as they have Trainium for training and Inferentia for inference. Hyperscalers obviously see better price/performance in this approach, and this is where the industry is headed. $AVGO and $MRVL are poised to be big winners, while $NVDA dominance will be under increasing pressure.

Oguz Erkan

223,244 views • 2 months ago

Electronics in Space Philip Johnston Reusable rockets are about to dramatically increase humanity's capacity to put things in space, and that means an enormous new market for compute built to operate there. We want to see inference chips optimized for mass, thermal, and radiation tolerance, built for a world where space is no longer out of reach.

Electronics in Space Philip Johnston Reusable rockets are about to dramatically increase humanity's capacity to put things in space, and that means an enormous new market for compute built to operate there. We want to see inference chips optimized for mass, thermal, and radiation tolerance, built for a world where space is no longer out of reach.

Y Combinator

87,872 views • 2 months ago

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.

Amazon Web Services

27,021 views • 4 months ago

We are introducing Felix. Felix is a purpose-built agent for high finance, designed for long-running, complex workflows and capable of producing decks, models, and documents end-to-end. Felix executes so you can focus where it matters.

We are introducing Felix. Felix is a purpose-built agent for high finance, designed for long-running, complex workflows and capable of producing decks, models, and documents end-to-end. Felix executes so you can focus where it matters.

Rogo

909,787 views • 2 months ago

NVIDIA Vera Rubin is in full production and arrives just in time for the next frontier of AI. The #NVIDIARubin platform uses extreme co-design across six new chips to accelerate agentic AI, advanced reasoning and massive-scale MoE model inference. NVIDIA Vera Rubin NVL72: Six new chips, one AI supercomputer — see how we built it.

NVIDIA Vera Rubin is in full production and arrives just in time for the next frontier of AI. The #NVIDIARubin platform uses extreme co-design across six new chips to accelerate agentic AI, advanced reasoning and massive-scale MoE model inference. NVIDIA Vera Rubin NVL72: Six new chips, one AI supercomputer — see how we built it.

NVIDIA Data Center

27,643 views • 6 months ago

$SPCX spending $60B on Cursor validates the idea that real AI value accrues to the application layer that owns the workflow, proprietary data loop and enterprise distribution. $NOW is building exactly that as the orchestration layer for enterprise AI agents across business workflows. The Cursor deal is a reminder that the market will pay massive premiums for software that owns where work actually happens.

$SPCX spending $60B on Cursor validates the idea that real AI value accrues to the application layer that owns the workflow, proprietary data loop and enterprise distribution. $NOW is building exactly that as the orchestration layer for enterprise AI agents across business workflows. The Cursor deal is a reminder that the market will pay massive premiums for software that owns where work actually happens.

Shay Boloor

80,768 views • 25 days ago

Just watched a demo of Merge Agent Handler and this actually looks pretty useful if you’re building AI agents. Most agents can generate responses, but getting them to actually take actions across real tools is still messy. Integrations are one thing, but handling auth, permissions, and security usually makes it way more complicated than it should be. What I liked is that it feels built for real workflows — you can actually see what your agent is doing instead of just hoping it works. Feels like a practical layer between AI agents and real work getting done. Worth checking out:

Just watched a demo of Merge Agent Handler and this actually looks pretty useful if you’re building AI agents. Most agents can generate responses, but getting them to actually take actions across real tools is still messy. Integrations are one thing, but handling auth, permissions, and security usually makes it way more complicated than it should be. What I liked is that it feels built for real workflows — you can actually see what your agent is doing instead of just hoping it works. Feels like a practical layer between AI agents and real work getting done. Worth checking out:

Kawsar

31,812 views • 4 months ago

We are announcing new innovations across our datacenter fleet, including the latest AI optimized silicon from our industry partners and two new Microsoft-designed chips. #MSIgnite

We are announcing new innovations across our datacenter fleet, including the latest AI optimized silicon from our industry partners and two new Microsoft-designed chips. #MSIgnite

Microsoft

278,248 views • 2 years ago

AI agents are UNRELIABLE because you're giving them too much control. Instead of using a single prompt, structure your agent as a series of high level steps. In this demo, I'm running the same steps across two different websites. Good steps = good output. It's that simple!

AI agents are UNRELIABLE because you're giving them too much control. Instead of using a single prompt, structure your agent as a series of high level steps. In this demo, I'm running the same steps across two different websites. Good steps = good output. It's that simple!

Paul Klein IV

71,482 views • 1 year ago

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

MTS

44,057 views • 1 month ago

.Qualcomm and Microsoft are working together to help developers build for the age of AI agents, with deeper integration across Windows ML, native app experiences, and inference workflows.

.Qualcomm and Microsoft are working together to help developers build for the age of AI agents, with deeper integration across Windows ML, native app experiences, and inference workflows.

Microsoft Developer

11,578 views • 16 days ago

Folks are spending hundreds of dollars for AI agents that don't even do what they want. No longer. PRISM AI Agents is now available and open-source. Create your own agent in minutes, for free, directly from Github:

Folks are spending hundreds of dollars for AI agents that don't even do what they want. No longer. PRISM AI Agents is now available and open-source. Create your own agent in minutes, for free, directly from Github:

miles

126,894 views • 1 year ago

Introducing PAI 1.0: The First Version of Personal Super Intelligence PAI is the world's first context-aware, widget based AI agent that floats over your screen to read your context in real time and proactively suggest next steps, ensuring safety by executing actions only with your explicit approval. The floating AI interface designed to eliminate the prompt inequality and barrier. Structurally eliminating the "navigate to a chat window, write a prompt, explain your context" workflow that existing AI services have treated as a given for years.

Introducing PAI 1.0: The First Version of Personal Super Intelligence PAI is the world's first context-aware, widget based AI agent that floats over your screen to read your context in real time and proactively suggest next steps, ensuring safety by executing actions only with your explicit approval. The floating AI interface designed to eliminate the prompt inequality and barrier. Structurally eliminating the "navigate to a chat window, write a prompt, explain your context" workflow that existing AI services have treated as a given for years.

Eon (Yulheon) Seung

130,825 views • 1 month ago

This is NVIDIA Rubin. Six new chips designed to deliver one incredible AI supercomputer. Built with extreme codesign across compute, networking, and software, Rubin sets a new standard for building and deploying the most advanced AI systems at the lowest possible cost. #CES2026

This is NVIDIA Rubin. Six new chips designed to deliver one incredible AI supercomputer. Built with extreme codesign across compute, networking, and software, Rubin sets a new standard for building and deploying the most advanced AI systems at the lowest possible cost. #CES2026

NVIDIA Newsroom

31,665 views • 6 months ago

$NVDA CEO Jensen Huang says GPUs are effectively sold out across the cloud with availability so tight that even renting older-generation chips has become difficult. Spot prices for GPU rentals are rising as a result which favors $NBIS, $IREN & $CIFR as demand pushes through limited capacity.

$NVDA CEO Jensen Huang says GPUs are effectively sold out across the cloud with availability so tight that even renting older-generation chips has become difficult. Spot prices for GPU rentals are rising as a result which favors $NBIS, $IREN & $CIFR as demand pushes through limited capacity.

Shay Boloor

212,563 views • 5 months ago

$AMZN CEO: “If you are building a big inference business and want decent margings, not having your custom chips is a disadvantage.” 40% of $AMZN compute comes from its custom chips. Result? AWS margins climb higher while peers lag, according to SemiAnalysis.

$AMZN CEO: “If you are building a big inference business and want decent margings, not having your custom chips is a disadvantage.” 40% of $AMZN compute comes from its custom chips. Result? AWS margins climb higher while peers lag, according to SemiAnalysis.

Oguz Erkan

203,673 views • 1 month ago

EIP-8004 is coming to the Nova architecture, a trustless infrastructure for AI agents that introduces key on-chain registries, enabling agents to interact safely across the Shido Network. These core components allow autonomous AI agents to verify identity, build reputation, and collaborate without relying on a centralized platform. The result is a decentralized trust layer for agent-to-agent economies, where agents can autonomously discover, evaluate, and work with one another across the Shido ecosystem.

EIP-8004 is coming to the Nova architecture, a trustless infrastructure for AI agents that introduces key on-chain registries, enabling agents to interact safely across the Shido Network. These core components allow autonomous AI agents to verify identity, build reputation, and collaborate without relying on a centralized platform. The result is a decentralized trust layer for agent-to-agent economies, where agents can autonomously discover, evaluate, and work with one another across the Shido ecosystem.

Shido

390,734 views • 4 months ago