Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

AI’s next bottleneck isn’t just compute. It’s how efficiently we use memory. At GSA, Patrick Moorhead asks Samsung Semiconductor US President Paul Cho what AI needs to get right to scale through 2031. Cho points to deeper memory hierarchies and tighter co-design across memory, GPUs/XPUs, networking + software.

TheSixFiveMedia's profile picture

Six Five Media

4,547 subscribers

28,519 Aufrufe • vor 1 Monat •via X (Twitter)

Wissenschaft & TechnologieNachrichten & Politik
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

AI frameworks prioritize FLOPs and TOPS. But Samsung Semiconductor US President Paul Cho says the bottleneck has shifted. Memory bandwidth now determines AI inference performance and cost. He believes memory-centric compute could define AI’s next decade. Watch:
1:11

AI frameworks prioritize FLOPs and TOPS. But Samsung Semiconductor US President Paul Cho says the bottleneck has shifted. Memory bandwidth now determines AI inference performance and cost. He believes memory-centric compute could define AI’s next decade. Watch:

Six Five Media

19,685 Aufrufe • vor 1 Monat

Everyone’s still talking about GPUs. Meanwhile, memory is now the real constraint for AI at scale. From GTC, Patrick Moorhead & Daniel Newman are joined by Paul Cho & Indong Kim from Samsung Semiconductor US to discuss why bandwidth, not compute, is starting to define AI performance.
18:14

Everyone’s still talking about GPUs. Meanwhile, memory is now the real constraint for AI at scale. From GTC, Patrick Moorhead & Daniel Newman are joined by Paul Cho & Indong Kim from Samsung Semiconductor US to discuss why bandwidth, not compute, is starting to define AI performance.

Six Five Media

56,637 Aufrufe • vor 7 Monaten

AI scaling is running into some hard limits. Primarily memory, bandwidth, and power. Samsung Semiconductor US President Paul Cho joined Patrick Moorhead and Daniel Newman to unpack how the industry is responding at The Six Five Summit: AI Unleashed 2026. Watch on demand:
2:22

AI scaling is running into some hard limits. Primarily memory, bandwidth, and power. Samsung Semiconductor US President Paul Cho joined Patrick Moorhead and Daniel Newman to unpack how the industry is responding at The Six Five Summit: AI Unleashed 2026. Watch on demand:

Six Five Media

27,776 Aufrufe • vor 2 Monaten

AI demand is exploding, but scaling it isn’t just about more compute. At NVIDIA GTC, Samsung Semiconductor US President Paul Cho shares where the real innovation is happening. Samsung’s HBM4 shows that the next phase of AI performance will be driven by memory, integration, and system design. And the companies solving those layers will define what AI can actually do.
1:23

AI demand is exploding, but scaling it isn’t just about more compute. At NVIDIA GTC, Samsung Semiconductor US President Paul Cho shares where the real innovation is happening. Samsung’s HBM4 shows that the next phase of AI performance will be driven by memory, integration, and system design. And the companies solving those layers will define what AI can actually do.

Six Five Media

1,208,573 Aufrufe • vor 7 Monaten

$SKHY is the most direct public bet on HBM with its $NVDA exposure and margins more tightly tied to the memory bottleneck than $MU or Samsung. Samsung offers memory scale with conglomerate noise, Micron offers clean U.S. memory exposure while SK Hynix provides the strongest operating leverage to rising HBM content across Nvidia’s AI rack roadmap.
1:14

$SKHY is the most direct public bet on HBM with its $NVDA exposure and margins more tightly tied to the memory bottleneck than $MU or Samsung. Samsung offers memory scale with conglomerate noise, Micron offers clean U.S. memory exposure while SK Hynix provides the strongest operating leverage to rising HBM content across Nvidia’s AI rack roadmap.

Shay Boloor

185,147 Aufrufe • vor 3 Monaten

📈 NVIDIA tops AI leaderboards and benchmarks with open models driven by extreme co-design across compute, networking, memory, storage, and software. This includes models for biology, AI physics, agentic AI, physical AI, robotics, and autonomous vehicles. By being vertically integrated and horizontally open, NVIDIA enables its ecosystem to scale agentic workflows at every layer of the stack with the lowest cost per token.
0:38

📈 NVIDIA tops AI leaderboards and benchmarks with open models driven by extreme co-design across compute, networking, memory, storage, and software. This includes models for biology, AI physics, agentic AI, physical AI, robotics, and autonomous vehicles. By being vertically integrated and horizontally open, NVIDIA enables its ecosystem to scale agentic workflows at every layer of the stack with the lowest cost per token.

NVIDIA

85,929 Aufrufe • vor 6 Monaten

How do you design a chip years in advance when AI models are changing every six months? At GSA, Patrick Moorhead is moderating a panel with leaders from Samsung Semiconductor US, OpenAI, Broadcom, and AMD on how the industry plans architectures, packaging, and silicon amid rapid AI change. Stay tuned for coverage and insights from Global Semiconductor Alliance (GSA)
1:08

How do you design a chip years in advance when AI models are changing every six months? At GSA, Patrick Moorhead is moderating a panel with leaders from Samsung Semiconductor US, OpenAI, Broadcom, and AMD on how the industry plans architectures, packaging, and silicon amid rapid AI change. Stay tuned for coverage and insights from Global Semiconductor Alliance (GSA)

Six Five Media

23,480 Aufrufe • vor 1 Monat

Moonshot AI limiting Kimi K3 subscriptions because of compute constraints is a massive signal because the 2.8 trillion parameter model is already trained and the bottleneck is now emerging as usage scales. Serving that demand requires a ton more accelerators, enormous memory capacity and faster networking to keep data moving efficiently across the cluster. So as AI spending shifts toward serving millions or billions of daily requests then memory and networking become critical bottlenecks giving $MU, $SKHY, $AVGO, $CRDO, $ALAB and $MRVL direct exposure to inference at scale.
0:42

Moonshot AI limiting Kimi K3 subscriptions because of compute constraints is a massive signal because the 2.8 trillion parameter model is already trained and the bottleneck is now emerging as usage scales. Serving that demand requires a ton more accelerators, enormous memory capacity and faster networking to keep data moving efficiently across the cluster. So as AI spending shifts toward serving millions or billions of daily requests then memory and networking become critical bottlenecks giving $MU, $SKHY, $AVGO, $CRDO, $ALAB and $MRVL direct exposure to inference at scale.

Shay Boloor

123,791 Aufrufe • vor 3 Monaten

$NVDA hand-delivered the first Vera CPUs to Anthropic, OpenAI, SpaceX and Oracle Cloud. Vera is the company’s first custom CPU built for agentic AI marking its push deeper into the full compute stack as demand expands across CPUs, GPUs, networking and memory.
0:41

$NVDA hand-delivered the first Vera CPUs to Anthropic, OpenAI, SpaceX and Oracle Cloud. Vera is the company’s first custom CPU built for agentic AI marking its push deeper into the full compute stack as demand expands across CPUs, GPUs, networking and memory.

Shay Boloor

168,875 Aufrufe • vor 5 Monaten

Introducing Memory 101 🧠 Every ai tool you use is a goldfish. brilliant for one chat, close the tab and it forgets you completely. memory is the most underbuilt layer in ai right now. so we're explaining how it actually works, one episode at a time. coming up: → rag vs memory, why most "ai memory" is just search → the 5 things real memory needs: facts, profiles, graphs, time, forgetting → why good memory has to forget things follow along!
0:50

Introducing Memory 101 🧠 Every ai tool you use is a goldfish. brilliant for one chat, close the tab and it forgets you completely. memory is the most underbuilt layer in ai right now. so we're explaining how it actually works, one episode at a time. coming up: → rag vs memory, why most "ai memory" is just search → the 5 things real memory needs: facts, profiles, graphs, time, forgetting → why good memory has to forget things follow along!

supermemory

17,235 Aufrufe • vor 4 Monaten

Kimi K3 full fine-tuning is live on AC2. Our memory optimizations reduced GPUs required per training replica by ~40%. At nearly 3T parameters, Kimi forced us to rethink how we manage memory, communication, rollouts, and checkpoints. The result is a much more efficient path to training frontier-scale open models.
0:21

Kimi K3 full fine-tuning is live on AC2. Our memory optimizations reduced GPUs required per training replica by ~40%. At nearly 3T parameters, Kimi forced us to rethink how we manage memory, communication, rollouts, and checkpoints. The result is a much more efficient path to training frontier-scale open models.

Applied Compute

236,858 Aufrufe • vor 2 Monaten

What's the next big trade in AI infrastructure after memory? We asked Meltem Demirors and Kelly Greer: "Memory is ripping now. We think the next narrative is gonna be around networking." "Jensen is doing the exact same thing now that he did with Mellanox in 2019 that turned NVIDIA from a GPU company into a systems and networking company. He's doing it again with Marvell." "Hyperscalers are literally telling you the two biggest bottlenecks for AI spend right now are networking and memory." "The tea leaves are written literally everywhere... the market's just been on the memory track. We're trying to get ahead of that."
2:51

What's the next big trade in AI infrastructure after memory? We asked Meltem Demirors and Kelly Greer: "Memory is ripping now. We think the next narrative is gonna be around networking." "Jensen is doing the exact same thing now that he did with Mellanox in 2019 that turned NVIDIA from a GPU company into a systems and networking company. He's doing it again with Marvell." "Hyperscalers are literally telling you the two biggest bottlenecks for AI spend right now are networking and memory." "The tea leaves are written literally everywhere... the market's just been on the memory track. We're trying to get ahead of that."

MTS

35,125 Aufrufe • vor 5 Monaten

The future of AI depends on how compute, storage, networking, and software work together. Junchen Jiang, Co-founder of Tensormesh, on what it takes to build the next generation of AI infrastructure.
0:59

The future of AI depends on how compute, storage, networking, and software work together. Junchen Jiang, Co-founder of Tensormesh, on what it takes to build the next generation of AI infrastructure.

AI at AMD

116,017 Aufrufe • vor 4 Monaten

Introducing Dreaming: Across sessions, Devin builds a memory graph of how you like to work At night Devin self-improves its memory to remove stale records and discover latent information We are creating an OSS standard called Agent Memory Repo:
0:34

Introducing Dreaming: Across sessions, Devin builds a memory graph of how you like to work At night Devin self-improves its memory to remove stale records and discover latent information We are creating an OSS standard called Agent Memory Repo:

Cognition

634,866 Aufrufe • vor 1 Monat

$MU CEO Sanjay Mehrotra says AI is turning memory from a commodity into “strategic infrastructure.” AI cannot scale without more HBM DRAM and SSD capacity which is making memory far more valuable across the entire compute stack.
0:42

$MU CEO Sanjay Mehrotra says AI is turning memory from a commodity into “strategic infrastructure.” AI cannot scale without more HBM DRAM and SSD capacity which is making memory far more valuable across the entire compute stack.

Shay Boloor

194,460 Aufrufe • vor 2 Monaten

SK Hynix's Hoshik Kim admits high bandwidth memory "is not the answer" to the memory-wall problem, and SK is working on different solutions to solve the bottleneck. "No, actually, I kind of agree with Pat. So HBM is not the answer." [SK Hynix is admitting that HBM is a bad memory?] "Nothing is a bad memory. It's a good memory, but it's not the final answer. So actually, HBM will not solve the memory-wall problem. Memory-wall problem is an inherent AI problem, which you cannot avoid." "So we try our best to provide higher bandwidth with this memory, with HBM technology even though it has some limitation, as Pat already mentioned, with thermal, IOs, cost, a lot of stuff. "But that's the only thing we got right now to help ease this memory-wall problem, memory-bottleneck problem. But at the same time, we are working in different directions to solve this memory-wall problem with different solutions."
0:51

SK Hynix's Hoshik Kim admits high bandwidth memory "is not the answer" to the memory-wall problem, and SK is working on different solutions to solve the bottleneck. "No, actually, I kind of agree with Pat. So HBM is not the answer." [SK Hynix is admitting that HBM is a bad memory?] "Nothing is a bad memory. It's a good memory, but it's not the final answer. So actually, HBM will not solve the memory-wall problem. Memory-wall problem is an inherent AI problem, which you cannot avoid." "So we try our best to provide higher bandwidth with this memory, with HBM technology even though it has some limitation, as Pat already mentioned, with thermal, IOs, cost, a lot of stuff. "But that's the only thing we got right now to help ease this memory-wall problem, memory-bottleneck problem. But at the same time, we are working in different directions to solve this memory-wall problem with different solutions."

Fireside Alpha

161,722 Aufrufe • vor 3 Monaten

Kimi K3 is too large to run efficiently on a single accelerator so the workload must be distributed across many chips that exchange data at extremely high speeds. $CRDO and $ALAB benefit from the high speed connectivity inside AI clusters while $MRVL and $AVGO provide the networking, optical and custom silicon infrastructure required to scale them. Thats why inference constraints are bullish for the entire surrounding system including memory, networking, optics, switches, power, cooling and orchestration software.
0:42

Kimi K3 is too large to run efficiently on a single accelerator so the workload must be distributed across many chips that exchange data at extremely high speeds. $CRDO and $ALAB benefit from the high speed connectivity inside AI clusters while $MRVL and $AVGO provide the networking, optical and custom silicon infrastructure required to scale them. Thats why inference constraints are bullish for the entire surrounding system including memory, networking, optics, switches, power, cooling and orchestration software.

Shay Boloor

84,649 Aufrufe • vor 3 Monaten

$AAOI fits into AI buildout as data centers become massive networking machines where GPUs create nonstop traffic across racks , clusters & memory layers. As AI workloads move toward 800G & 1.6T connections, AAOI’s optical transceivers put it in middle of a major AI bottleneck.
0:56

$AAOI fits into AI buildout as data centers become massive networking machines where GPUs create nonstop traffic across racks , clusters & memory layers. As AI workloads move toward 800G & 1.6T connections, AAOI’s optical transceivers put it in middle of a major AI bottleneck.

Shay Boloor

228,987 Aufrufe • vor 6 Monaten

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!
1:37

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!

Andrew Ng

163,155 Aufrufe • vor 7 Monaten