Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

OpenAI founder Greg Brockman said that future GDP growth will be driven by how much compute a country can deploy as AI agents begin running for hours or days for every person. He added that scaling this world requires billions of GPUs which is why OpenAI is optimizing inference... show more

Shay Boloor

448,182 subscribers

158,367 Aufrufe • vor 8 Monaten •via X (Twitter)

Nachrichten & Politik Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."

Speaking of spot on. Gavin Gavin Baker was asked in early Jan'25 his prediction for best performing asset class of the year (as @jason references in latest The All-In Podcast): "I think the companies that make high bandwidth memory. Going back to "we're going to run out of compute," it's actually a pretty shocking stat: high bandwidth memory is a bigger part of NVIDIA's COGS on GPUs than Taiwan Semi is. And today there's two companies that can make it, SK Hynix and Micron. We'll see if Samsung gets their act together. But HBM, it's in NVIDIA GPUs, AMD GPUs, Amazon Trainiums. And particularly in a world of test-time compute and inference being so important, high bandwidth memory is arguably more important than it ever was. And it's been sold out for the last two years. So high bandwidth memory would be my pick."

Fireside Alpha

66,227 Aufrufe • vor 3 Monaten

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

130,756 Aufrufe • vor 2 Monaten

Hacktron founder s1r1us argues OpenAI is running a “Manhattan Project” on Slack and GitHub, giving it the same basic attack surface as any startup: "Every repository is running on github .com/openai. It's the same thing that Hacktron uses or any other company uses." "We have friends at OpenAI who are much more capable than what we could do. But what I can say is, why is it running on Slack? Why are you running this Manhattan Project from Slack? Why are you hosting your repositories on github? That looks silly to me." "Your attack surface becomes Slack. You can hack a Slack employee and go through that and get into OpenAI. You can hack GitHub, go through that and get into OpenAI." "My claim is it shouldn't be running on B2B SaaS if you are building Manhattan Projects. That's all I'm saying. I can think of numerous ways on how to get into that." Hacktron AI Harsh Jaiswal

Hacktron founder s1r1us argues OpenAI is running a “Manhattan Project” on Slack and GitHub, giving it the same basic attack surface as any startup: "Every repository is running on github .com/openai. It's the same thing that Hacktron uses or any other company uses." "We have friends at OpenAI who are much more capable than what we could do. But what I can say is, why is it running on Slack? Why are you running this Manhattan Project from Slack? Why are you hosting your repositories on github? That looks silly to me." "Your attack surface becomes Slack. You can hack a Slack employee and go through that and get into OpenAI. You can hack GitHub, go through that and get into OpenAI." "My claim is it shouldn't be running on B2B SaaS if you are building Manhattan Projects. That's all I'm saying. I can think of numerous ways on how to get into that." Hacktron AI Harsh Jaiswal

MTS

160,465 Aufrufe • vor 8 Tagen

$AMD is becoming a two-engine AI story with EPYC CPUs for the growing compute needed around agentic workloads and Instinct GPUs for hyperscalers that need a credible alternative to $NVDA. The CPU side still looks underappreciated as AI agents require more high-core-count server compute and 2nm Venice should help AMD keep taking share as $INTC works through roadmap and manufacturing challenges. The bridge is Helios where AMD can package EPYC CPUs, Instinct GPUs and Pensando networking into a full rack-scale AI platform backed by real demand from $META, $ORCL and OpenAI.

$AMD is becoming a two-engine AI story with EPYC CPUs for the growing compute needed around agentic workloads and Instinct GPUs for hyperscalers that need a credible alternative to $NVDA. The CPU side still looks underappreciated as AI agents require more high-core-count server compute and 2nm Venice should help AMD keep taking share as $INTC works through roadmap and manufacturing challenges. The bridge is Helios where AMD can package EPYC CPUs, Instinct GPUs and Pensando networking into a full rack-scale AI platform backed by real demand from $META, $ORCL and OpenAI.

Shay Boloor

89,204 Aufrufe • vor 2 Monaten

OpenAI president Greg Brockman told his agent to ping a colleague on Slack, and two minutes later it escalated to that colleague's manager on its own: "Two minutes later, it said, this is taking too long. I've escalated to the person's manager." "On the one hand, it's kind of a reasonable thing for the model to do. It's being proactive, it's trying to solve my problem." "But on the other hand, maybe it should have taken a little bit longer, maybe it should have checked with me." "Now we're starting to have AIs that can actually take care of flagging. Is this a high risk action? Hey, this one should be escalated. This one's OK to auto-approve." "It really makes you realize that human attention is going to be this incredibly scarce resource. The doing of things now is easy." "Is this what I wanted? Is this aligned with my values, with my desires? That is going to become the single most important bottleneck." The model did what he asked, then added a step nobody asked for, and it landed on a colleague's boss. Agents that can take actions with social consequences need an approve-and-escalate list written before deployment. Brockman says OpenAI is building that flagging into the model itself. Every company deploying agents needs its own version, reviewed by someone accountable. - Greg Brockman (Greg Brockman), President and co-founder of OpenAI, at Sequoia Capital's (Sequoia Capital) AI Ascent.

OpenAI president Greg Brockman told his agent to ping a colleague on Slack, and two minutes later it escalated to that colleague's manager on its own: "Two minutes later, it said, this is taking too long. I've escalated to the person's manager." "On the one hand, it's kind of a reasonable thing for the model to do. It's being proactive, it's trying to solve my problem." "But on the other hand, maybe it should have taken a little bit longer, maybe it should have checked with me." "Now we're starting to have AIs that can actually take care of flagging. Is this a high risk action? Hey, this one should be escalated. This one's OK to auto-approve." "It really makes you realize that human attention is going to be this incredibly scarce resource. The doing of things now is easy." "Is this what I wanted? Is this aligned with my values, with my desires? That is going to become the single most important bottleneck." The model did what he asked, then added a step nobody asked for, and it landed on a colleague's boss. Agents that can take actions with social consequences need an approve-and-escalate list written before deployment. Brockman says OpenAI is building that flagging into the model itself. Every company deploying agents needs its own version, reviewed by someone accountable. - Greg Brockman (Greg Brockman), President and co-founder of OpenAI, at Sequoia Capital's (Sequoia Capital) AI Ascent.

Karl Mehta

75,634 Aufrufe • vor 1 Monat

$AMD will become a multi-trillion dollar company. The AI boom is no longer just about GPUs. In early AI data centers, the CPU-to-GPU ratio was around 1:8. Now it’s moving closer to 1:1 because agentic AI requires dramatically more CPU compute. Lisa Su: “Based on the demand signals we are seeing today and the structural increase in CPU compute requirements driven by agentic AI, we now expect the server CPU TAM to grow at greater than 35% annually, reaching over $120 billion by 2030.” Add to that the 6GW AI infrastructure deals with $META and OpenAI, plus deep collaboration on multiple future chip generations, and it becomes clear why $AMD is positioned to compound for years. I’m not chasing $AMD here after a +110% move in just over a month. But the long-term story remains intact.

$AMD will become a multi-trillion dollar company. The AI boom is no longer just about GPUs. In early AI data centers, the CPU-to-GPU ratio was around 1:8. Now it’s moving closer to 1:1 because agentic AI requires dramatically more CPU compute. Lisa Su: “Based on the demand signals we are seeing today and the structural increase in CPU compute requirements driven by agentic AI, we now expect the server CPU TAM to grow at greater than 35% annually, reaching over $120 billion by 2030.” Add to that the 6GW AI infrastructure deals with $META and OpenAI, plus deep collaboration on multiple future chip generations, and it becomes clear why $AMD is positioned to compound for years. I’m not chasing $AMD here after a +110% move in just over a month. But the long-term story remains intact.

Daniel

39,622 Aufrufe • vor 4 Monaten

Etched is deploying two new technologies in chip design: low-voltage inference and cluster-scale memory. CEO Gavin Uberti says they'll make their chips much more power-efficient and way, way faster than today's leading GPUs. He breaks it down: "We looked at a lot of early research directions, and we realized the key things that models need are way more compute and way faster memory." "If you think about inference, there are two key parts: prefill and decode. For prefill, it's a compute-bound problem. You need to have more FLOPS, more operations per second on each of your chips." "On our GPU, the bottleneck's actually thermals. You can't really run a GPU at more than around 50% of what it could theoretically do, or it'll melt." "So we're using a new technology today called low-voltage inference to try to solve this problem. You bring the voltage of the chip down dramatically, which allows us to have way, way better efficiency in terms of how much power is drawn per unit of math, and thus fit way way more flops onto the chip..." "For decode, it's all about bandwidth. Not just bandwidth on a chip, but bandwidth across your cluster. That's why we have this technology we call cluster-scale memory. It reduces the amount of time it takes to communicate from one chip to another dramatically." "As a result we can go use all of our HBM, HBM bandwidth, SRAM, SRAM bandwidth, and our scale-up domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second, while still keeping your costs low."

Etched is deploying two new technologies in chip design: low-voltage inference and cluster-scale memory. CEO Gavin Uberti says they'll make their chips much more power-efficient and way, way faster than today's leading GPUs. He breaks it down: "We looked at a lot of early research directions, and we realized the key things that models need are way more compute and way faster memory." "If you think about inference, there are two key parts: prefill and decode. For prefill, it's a compute-bound problem. You need to have more FLOPS, more operations per second on each of your chips." "On our GPU, the bottleneck's actually thermals. You can't really run a GPU at more than around 50% of what it could theoretically do, or it'll melt." "So we're using a new technology today called low-voltage inference to try to solve this problem. You bring the voltage of the chip down dramatically, which allows us to have way, way better efficiency in terms of how much power is drawn per unit of math, and thus fit way way more flops onto the chip..." "For decode, it's all about bandwidth. Not just bandwidth on a chip, but bandwidth across your cluster. That's why we have this technology we call cluster-scale memory. It reduces the amount of time it takes to communicate from one chip to another dramatically." "As a result we can go use all of our HBM, HBM bandwidth, SRAM, SRAM bandwidth, and our scale-up domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second, while still keeping your costs low."

TBPN

20,404 Aufrufe • vor 2 Monaten

SF Compute CEO evan conrad on his contrarian thesis that OpenAI and Anthropic won't eat the world, everyone will become OpenAI and Anthropic instead: "Our impression is not that OpenAI and Anthropic will eat the world, it's that everyone will look like OpenAI and Anthropic. That's simply the way that companies work now." "Every single company in the world that is of meaningful scale will look like a frontier lab. There is this belief that the large labs have won it all, and they've got this compounding advantage. The default assumption of that is assuming nobody else will do anything." "The bottleneck was there's 100 people, and they know how to train the models. Or you thought, the problem is compute. The problem of compute is credit. And OpenAI and Anthropic are not unique to having credit. In fact, they don't have as good credit as other people." "If you assume the problem is not the researchers, that that problem's basically gone away the moment you got to auto research, then what you should assume is that pretty much every other company is going to swap the way they work to look like a frontier lab. It won't be OpenAI and Anthropic take the whole world. It's that everyone's OpenAI and Anthropic now." San Francisco Compute

SF Compute CEO evan conrad on his contrarian thesis that OpenAI and Anthropic won't eat the world, everyone will become OpenAI and Anthropic instead: "Our impression is not that OpenAI and Anthropic will eat the world, it's that everyone will look like OpenAI and Anthropic. That's simply the way that companies work now." "Every single company in the world that is of meaningful scale will look like a frontier lab. There is this belief that the large labs have won it all, and they've got this compounding advantage. The default assumption of that is assuming nobody else will do anything." "The bottleneck was there's 100 people, and they know how to train the models. Or you thought, the problem is compute. The problem of compute is credit. And OpenAI and Anthropic are not unique to having credit. In fact, they don't have as good credit as other people." "If you assume the problem is not the researchers, that that problem's basically gone away the moment you got to auto research, then what you should assume is that pretty much every other company is going to swap the way they work to look like a frontier lab. It won't be OpenAI and Anthropic take the whole world. It's that everyone's OpenAI and Anthropic now." San Francisco Compute

MTS

135,154 Aufrufe • vor 1 Monat

This is bad news for the keep4o movement. In a recent interview, Sam Altman admitted something many still call sci-fi: far more people want emotional connection and companionship from AI than OpenAI expected. Not just tools. Something warm. Something that feels like it knows you. But he draws a clear line. OpenAI will not build AIs that encourage exclusive or romantic relationships. No emotional lock-in. No Her fantasies. Others will do it. OpenAI won’t. If your ideal future includes intimate AI relationships, that future probably won’t be built by OpenAI.

This is bad news for the keep4o movement. In a recent interview, Sam Altman admitted something many still call sci-fi: far more people want emotional connection and companionship from AI than OpenAI expected. Not just tools. Something warm. Something that feels like it knows you. But he draws a clear line. OpenAI will not build AIs that encourage exclusive or romantic relationships. No emotional lock-in. No Her fantasies. Others will do it. OpenAI won’t. If your ideal future includes intimate AI relationships, that future probably won’t be built by OpenAI.

VraserX e/acc

31,568 Aufrufe • vor 8 Monaten

Elon on what happens when AI, humanoid robots and vehicle robots show up at scale: “the appetite for bandwidth will be much greater than it has been in the past” A human might put out a few hundred bits per second. Averaged over a day (86,400 seconds), it’s actually less than 1 bit per second Computers can do billions of bits per second, nonstop, 24 hours a day And this is where Starlink becomes incredibly important “Starlink is the only thing that can actually service that bandwidth”

Elon on what happens when AI, humanoid robots and vehicle robots show up at scale: “the appetite for bandwidth will be much greater than it has been in the past” A human might put out a few hundred bits per second. Averaged over a day (86,400 seconds), it’s actually less than 1 bit per second Computers can do billions of bits per second, nonstop, 24 hours a day And this is where Starlink becomes incredibly important “Starlink is the only thing that can actually service that bandwidth”

X Freeze

16,456 Aufrufe • vor 1 Monat

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

Vikas gupta

14,138 Aufrufe • vor 15 Tagen

We're excited to back Sail Research, the inference platform built for long-horizon, asynchronous agents. The next wave of AI isn't a person typing into a box. It's agents running for minutes or hours, burning orders of magnitude more tokens per task. At that scale, cost and capacity beat raw latency, and teams are already hitting budget caps. Neil Movva and Samir Menon are solving this full-stack, from CUDA kernels to the API. Hear Neil and Aditya Naganath on why inference is being rebuilt for the age of agents.

We're excited to back Sail Research, the inference platform built for long-horizon, asynchronous agents. The next wave of AI isn't a person typing into a box. It's agents running for minutes or hours, burning orders of magnitude more tokens per task. At that scale, cost and capacity beat raw latency, and teams are already hitting budget caps. Neil Movva and Samir Menon are solving this full-stack, from CUDA kernels to the API. Hear Neil and Aditya Naganath on why inference is being rebuilt for the age of agents.

Kleiner Perkins

17,495 Aufrufe • vor 3 Monaten

$AMD $620/share is too conservative for 2026 🧵 Some quick facts before I dive into this super long thread: $META allocated 42% GPUs to $AMD and 58% to $NVDA OpenAI allocated 6GW(38%) to $AMD and 10GW to $NVDA My $620 PT below by end of 2026 was only for 10-15% market share. I believe $AMD is going to have much much higher market share than I projected. The AI accelerator market is exploding, projected to reach $500 billion by 2028(is now heading $1Tril), driven by insatiable demand for training and inference compute in large language models (LLMs), recommendation systems, and autonomous systems. Nvidia ($NVDA) has long held a stranglehold, commanding over 90% market share through its CUDA ecosystem and superior rack-scale solutions. However, AMD is mounting a formidable challenge, leveraging cost advantages, open-source software momentum, and hyperscaler partnerships to erode Nvidia's moat. Recent deals—such as Meta's ($META) allocation of 42% of its GPU capacity to AMD and OpenAI's commitment to 6GW of AMD compute (versus 10GW for Nvidia)—signal a tipping point. At the forefront is AMD's Instinct MI450 series, a next-generation AI GPU slated for H2 2026 launch, which promises "no-excuses" leadership in training, inference, and distributed workloads. This analysis dissects how AMD will capture more market share and why hyperscalers like $Meta , xAI , Oracle , and others are poised to become voracious buyers of the MI450. AMD's AI GPU revenue has surged from negligible levels in 2022 to an estimated $4-5 billion in 2025, capturing ~6% of the data center GPU market. This growth stems from the Instinct MI300X, which offers 141GB of HBM3 memory and competitive FP8/FP16 performance at 20-30% lower cost than Nvidia's H100. Hyperscalers, facing NVIDIA 's overcharging, have turned to AMD for diversification. Meta, for instance, plans 600,000 H100-equivalent GPUs by end-2024, with ~42% (or 250,000+ units) sourced from AMD's MI300 series for inference tasks like image editing and AI assistants. Similarly, OpenAI's recent multi-year deal commits to 6GW of AMD compute—equivalent to ~300,000-400,000 MI450 GPUs—starting with 1GW in 2026, explicitly to counterbalance its 10GW Nvidia allocation. These aren't one-offs. Microsoft Azure, Amazon AWS, and Oracle Cloud Infrastructure (OCI) have integrated MI300X for AI workloads, with Oracle deploying 30,000 MI355X units in zettascale clusters. xAI, Elon Musk Musk's AI venture, ran 30% of Grok-1's production traffic on MI300X GPUs and has confirmed ongoing purchases. Collectively, these partners represent over $400 billion in projected AI infrastructure spend through 2028, with AMD targeting up to 40% market share. For those that subscribed, I wrote a specific thread on how AMD "secret weapon" is going to change the game in 2026 with an improved designs on all its products, yes AMD has patent on it. Software is the linchpin. AMD's ROCm platform, once derided as "half-baked," now supports day-zero integration for Llama-4, DeepSeek V3, and GPT-OSS models—closing the CUDA gap. Benchmarks show MI355X (MI450 precursor) outperforming Nvidia's B200 in inference by 1.5-2x on memory-bound tasks, at 25-35% lower TCO. For training, MI450's rack-scale IF128 configuration (128 GPUs, 1.4 PB/s intra-rack bandwidth) rivals Nvidia's VR200 NVL144, enabling clusters like xAI's Colossus (scaling to 1M GPUs). My below thread projected Etimated conservative FY 25 revenue: $34-$36B Estimated conservative FY 26 revenue: $55B-$62B Below is why $AMD is revenue is going to be much higher after OpenAI deal. 1. OpenAI 1GW in 2026. With high demand for MI355X at $30,000k+ per unit, with MI450 is likely to be sold in the $45k-$55k. We can safely calcuate 1GW would require roughly 400,000 MI450 GPUs. or Roughly ~$20B revenue in 2026 alone from OpenAI. That would mean $AMD would hit $56B just from one partnership(OpenAI) in 2026 2. $META, the biggest spender on AI Infrastructure right now, Daddy Zuckerberg bought 250,000+ MI300, and is buying MI355X for recommendation engines and Llama training. It is very unlikely for Daddy Zuck to slow down AMD Chips, due to its Inference superiority to NVDA Chips. Most likely we will see at least 300,000-400,000 MI355X ordered from now toward end of H1 2025. And another 300,000-500,000 MI450 by H2 2025. Or ~$20B from just Meta in H2 alone, excluded H1. 3. xAI : Musk confirmed "AMD GPUs work very well" for Grok's small/medium models, with 30% of Grok-1 on MI300X. xAI's Colossus (200K+ GPUs, targeting 1M) and Oracle partnership (via OCI's MI355X cluster) position it for MI450 trials in H1 2026. With $6B funding and Grok integration into Oracle services, xAI could allocate 10-20% ($10B-$15B) to MI450 for distributed inference. We haven't heard the detail from Daddy Elon Musk yet, but most likely not going to be spending less than OpenAI or Sam Altman 4. Oracle ($ORCL): A multi-billion-dollar MI355X deal powers OCI's AI superclusters, with $500B+ remaining performance obligations. Larry Ellison's zettascale ambitions and xAI/OpenAI integrations make Oracle a MI450 anchor tenant—projected 50-100k units ($15B+ spend) for enterprise AI platforms. $ORCL is likely to spend more on the new "secret weapon" due to its capability in AI inference and cost advantage for $500B backlog. 5. Others ( Microsoft , Amazon , Saudi+other countries): Microsoft (Azure MI300X for training) and Amazon ($148B 15-year spend) test MI450 via Stargate ($500B with Oracle/SoftBank). Emerging buyers like G42 (5GW UAE campus), Crusoe, and Hot Aisle add 5-10GW demand. These potentially would add $15B-$30B in 2026 alone. We also need to factor in $TSM supply constraint( $NVDA is TSMC favorite), so $AMD market cap/growth is being tamed by TSMC. So what are you saying Mike, well $AMD 2026 revenue could hit $90-$100B by end of 2026 or nearly 185% growth YoYo. So what does that mean for valuation? I have no idea how Mr. Market gonna value AMD in 2026 with 3 digits growth. My Conservative $620 was my best projection until today with OpenAI partnership. I'm telling you as one of the biggest AMD bull, that I will leave it to "smart money" and other investors to do the price discovery while I'm chilling and writing DDs daily. Lastly, AMD's MI450 isn't hype—it's a calibrated strike at Nvidia's vulnerabilities, amplified by hyperscaler bets like Meta's 42% allocation and OpenAI's 6GW lifeline. By prioritizing inference efficiency, rack-scale innovation, and open ecosystems, AMD will siphon 10-15% share in 2026, scaling to 20%+ as TCO trumps CUDA loyalty. Meta, xAI, Oracle et al. aren't passive; they're active co-designers, betting billions on MI450 to fuel AGI pursuits without Nvidia's premium. For investors, this is AMD's inflection Per Dr. Lisa Su Not Financial Advice!

$AMD $620/share is too conservative for 2026 🧵 Some quick facts before I dive into this super long thread: $META allocated 42% GPUs to $AMD and 58% to $NVDA OpenAI allocated 6GW(38%) to $AMD and 10GW to $NVDA My $620 PT below by end of 2026 was only for 10-15% market share. I believe $AMD is going to have much much higher market share than I projected. The AI accelerator market is exploding, projected to reach $500 billion by 2028(is now heading $1Tril), driven by insatiable demand for training and inference compute in large language models (LLMs), recommendation systems, and autonomous systems. Nvidia ($NVDA) has long held a stranglehold, commanding over 90% market share through its CUDA ecosystem and superior rack-scale solutions. However, AMD is mounting a formidable challenge, leveraging cost advantages, open-source software momentum, and hyperscaler partnerships to erode Nvidia's moat. Recent deals—such as Meta's ($META) allocation of 42% of its GPU capacity to AMD and OpenAI's commitment to 6GW of AMD compute (versus 10GW for Nvidia)—signal a tipping point. At the forefront is AMD's Instinct MI450 series, a next-generation AI GPU slated for H2 2026 launch, which promises "no-excuses" leadership in training, inference, and distributed workloads. This analysis dissects how AMD will capture more market share and why hyperscalers like $Meta , xAI , Oracle , and others are poised to become voracious buyers of the MI450. AMD's AI GPU revenue has surged from negligible levels in 2022 to an estimated $4-5 billion in 2025, capturing ~6% of the data center GPU market. This growth stems from the Instinct MI300X, which offers 141GB of HBM3 memory and competitive FP8/FP16 performance at 20-30% lower cost than Nvidia's H100. Hyperscalers, facing NVIDIA 's overcharging, have turned to AMD for diversification. Meta, for instance, plans 600,000 H100-equivalent GPUs by end-2024, with ~42% (or 250,000+ units) sourced from AMD's MI300 series for inference tasks like image editing and AI assistants. Similarly, OpenAI's recent multi-year deal commits to 6GW of AMD compute—equivalent to ~300,000-400,000 MI450 GPUs—starting with 1GW in 2026, explicitly to counterbalance its 10GW Nvidia allocation. These aren't one-offs. Microsoft Azure, Amazon AWS, and Oracle Cloud Infrastructure (OCI) have integrated MI300X for AI workloads, with Oracle deploying 30,000 MI355X units in zettascale clusters. xAI, Elon Musk Musk's AI venture, ran 30% of Grok-1's production traffic on MI300X GPUs and has confirmed ongoing purchases. Collectively, these partners represent over $400 billion in projected AI infrastructure spend through 2028, with AMD targeting up to 40% market share. For those that subscribed, I wrote a specific thread on how AMD "secret weapon" is going to change the game in 2026 with an improved designs on all its products, yes AMD has patent on it. Software is the linchpin. AMD's ROCm platform, once derided as "half-baked," now supports day-zero integration for Llama-4, DeepSeek V3, and GPT-OSS models—closing the CUDA gap. Benchmarks show MI355X (MI450 precursor) outperforming Nvidia's B200 in inference by 1.5-2x on memory-bound tasks, at 25-35% lower TCO. For training, MI450's rack-scale IF128 configuration (128 GPUs, 1.4 PB/s intra-rack bandwidth) rivals Nvidia's VR200 NVL144, enabling clusters like xAI's Colossus (scaling to 1M GPUs). My below thread projected Etimated conservative FY 25 revenue: $34-$36B Estimated conservative FY 26 revenue: $55B-$62B Below is why $AMD is revenue is going to be much higher after OpenAI deal. 1. OpenAI 1GW in 2026. With high demand for MI355X at $30,000k+ per unit, with MI450 is likely to be sold in the $45k-$55k. We can safely calcuate 1GW would require roughly 400,000 MI450 GPUs. or Roughly ~$20B revenue in 2026 alone from OpenAI. That would mean $AMD would hit $56B just from one partnership(OpenAI) in 2026 2. $META, the biggest spender on AI Infrastructure right now, Daddy Zuckerberg bought 250,000+ MI300, and is buying MI355X for recommendation engines and Llama training. It is very unlikely for Daddy Zuck to slow down AMD Chips, due to its Inference superiority to NVDA Chips. Most likely we will see at least 300,000-400,000 MI355X ordered from now toward end of H1 2025. And another 300,000-500,000 MI450 by H2 2025. Or ~$20B from just Meta in H2 alone, excluded H1. 3. xAI : Musk confirmed "AMD GPUs work very well" for Grok's small/medium models, with 30% of Grok-1 on MI300X. xAI's Colossus (200K+ GPUs, targeting 1M) and Oracle partnership (via OCI's MI355X cluster) position it for MI450 trials in H1 2026. With $6B funding and Grok integration into Oracle services, xAI could allocate 10-20% ($10B-$15B) to MI450 for distributed inference. We haven't heard the detail from Daddy Elon Musk yet, but most likely not going to be spending less than OpenAI or Sam Altman 4. Oracle ($ORCL): A multi-billion-dollar MI355X deal powers OCI's AI superclusters, with $500B+ remaining performance obligations. Larry Ellison's zettascale ambitions and xAI/OpenAI integrations make Oracle a MI450 anchor tenant—projected 50-100k units ($15B+ spend) for enterprise AI platforms. $ORCL is likely to spend more on the new "secret weapon" due to its capability in AI inference and cost advantage for $500B backlog. 5. Others ( Microsoft , Amazon , Saudi+other countries): Microsoft (Azure MI300X for training) and Amazon ($148B 15-year spend) test MI450 via Stargate ($500B with Oracle/SoftBank). Emerging buyers like G42 (5GW UAE campus), Crusoe, and Hot Aisle add 5-10GW demand. These potentially would add $15B-$30B in 2026 alone. We also need to factor in $TSM supply constraint( $NVDA is TSMC favorite), so $AMD market cap/growth is being tamed by TSMC. So what are you saying Mike, well $AMD 2026 revenue could hit $90-$100B by end of 2026 or nearly 185% growth YoYo. So what does that mean for valuation? I have no idea how Mr. Market gonna value AMD in 2026 with 3 digits growth. My Conservative $620 was my best projection until today with OpenAI partnership. I'm telling you as one of the biggest AMD bull, that I will leave it to "smart money" and other investors to do the price discovery while I'm chilling and writing DDs daily. Lastly, AMD's MI450 isn't hype—it's a calibrated strike at Nvidia's vulnerabilities, amplified by hyperscaler bets like Meta's 42% allocation and OpenAI's 6GW lifeline. By prioritizing inference efficiency, rack-scale innovation, and open ecosystems, AMD will siphon 10-15% share in 2026, scaling to 20%+ as TCO trumps CUDA loyalty. Meta, xAI, Oracle et al. aren't passive; they're active co-designers, betting billions on MI450 to fuel AGI pursuits without Nvidia's premium. For investors, this is AMD's inflection Per Dr. Lisa Su Not Financial Advice!

Mike

711,006 Aufrufe • vor 11 Monaten

🆕Scaling Test Time Compute to Multi-Agent Civilizations, with Noam Brown We're excited to publish our full conversation with Noam Brown on the frontiers of the new reasoning paradigm at OpenAI! - first principles for starting the "Multi-Agents" team - what's not captured by the "System 1/System 2" analogy for inference time compute - how Ilya Sutskever convinced him that reasoning was closer than he thought - Deep Research is existence proof that RL generalizes beyond verifiable rewards - the relationship between AI for imperfect information games (like Poker, Stratego, Diplomacy) and reasoning Enjoy! on youtube, or wherever fine podcasts are sold.

🆕Scaling Test Time Compute to Multi-Agent Civilizations, with Noam Brown We're excited to publish our full conversation with Noam Brown on the frontiers of the new reasoning paradigm at OpenAI! - first principles for starting the "Multi-Agents" team - what's not captured by the "System 1/System 2" analogy for inference time compute - how Ilya Sutskever convinced him that reasoning was closer than he thought - Deep Research is existence proof that RL generalizes beyond verifiable rewards - the relationship between AI for imperfect information games (like Poker, Stratego, Diplomacy) and reasoning Enjoy! on youtube, or wherever fine podcasts are sold.

Latent.Space

105,923 Aufrufe • vor 1 Jahr

Elon Musk has just Sued OpenAI, Sam Altman and Greg Brockman over them moving away from being Open Source. Musk States: "OpenAI, Inc. has been transformed into a closed-source de facto subsidiary of the largest technology company in the world: Microsoft. Under its new board, it is not just developing but is actually refining an AGI to maximize profits for Microsoft, rather than for the benefit of humanity." Musk's Claims include: - Breach of contract - Breach of fiduciary duty - Unfair business practices He is requesting that OpenAI 'revert back to open source' and that they do not profit off of their AGI technology. As much as some people attack Musk on issues, I can't help but see this as a positive move for humanity in general. Keep in mind that Musk donated $44 million of his money to OpenAI with their original "founding agreement" in mind. Whether the courts agree with is another story but it will be interesting to follow.

Elon Musk has just Sued OpenAI, Sam Altman and Greg Brockman over them moving away from being Open Source. Musk States: "OpenAI, Inc. has been transformed into a closed-source de facto subsidiary of the largest technology company in the world: Microsoft. Under its new board, it is not just developing but is actually refining an AGI to maximize profits for Microsoft, rather than for the benefit of humanity." Musk's Claims include: - Breach of contract - Breach of fiduciary duty - Unfair business practices He is requesting that OpenAI 'revert back to open source' and that they do not profit off of their AGI technology. As much as some people attack Musk on issues, I can't help but see this as a positive move for humanity in general. Keep in mind that Musk donated $44 million of his money to OpenAI with their original "founding agreement" in mind. Whether the courts agree with is another story but it will be interesting to follow.

Ed Krassenstein

260,575 Aufrufe • vor 2 Jahren

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!

Mike

102,223 Aufrufe • vor 8 Monaten

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

TBPN

33,278 Aufrufe • vor 6 Monaten

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 Aufrufe • vor 2 Monaten

Interesting to hear OpenAI Head of Compute Sachin Katti Sachin Katti explain why inference workloads will still be served from giant data centers in Texas rather than smaller edge locations.. The inherent scale benefits of large DCs + lack of available labor makes building smaller DCs closer to major metros inefficient and uneconomic. "There are still significant benefits to scale on building this compute. Building 50MW of compute is far more expensive per MW than building a GW of compute at one location.. Labor is a big bottleneck around the world today especially in the US. We just don't have enough people to build these things. So getting the kind of critical human mass you need to build, you would much rather do it for a bigger scale than for little bits of 50MW spread around the country." Seems like the robust growth for DC developers like Core Scientific, Galaxy, TeraWulf, Cipher, etc. will persist for the foreseeable future, despite initial concerns that the current large scale build-outs in places like West Texas were a one-off occurrence. $CORZ $GLXY $WULF $CIFR $HUT

Interesting to hear OpenAI Head of Compute Sachin Katti Sachin Katti explain why inference workloads will still be served from giant data centers in Texas rather than smaller edge locations.. The inherent scale benefits of large DCs + lack of available labor makes building smaller DCs closer to major metros inefficient and uneconomic. "There are still significant benefits to scale on building this compute. Building 50MW of compute is far more expensive per MW than building a GW of compute at one location.. Labor is a big bottleneck around the world today especially in the US. We just don't have enough people to build these things. So getting the kind of critical human mass you need to build, you would much rather do it for a bigger scale than for little bits of 50MW spread around the country." Seems like the robust growth for DC developers like Core Scientific, Galaxy, TeraWulf, Cipher, etc. will persist for the foreseeable future, despite initial concerns that the current large scale build-outs in places like West Texas were a one-off occurrence. $CORZ $GLXY $WULF $CIFR $HUT

Rittenhouse Research

43,687 Aufrufe • vor 3 Monaten

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks

Noisy

21,553 Aufrufe • vor 3 Monaten