Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Crusoe Managed Inference is now available. Run model inference faster, easier, and more cost-effectively. Large-context AI solutions demand fast outputs, massive throughput, and resilient scaling. Crusoe Managed Inference delivers breakthrough speed & reduced input-token spend.

Crusoe

14,933 subscribers

12,096,815 views • 10 months ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

.Kimi.ai K3 from Moonshot AI is now live on DigitalOcean Inference Engine. 1M-token context, native vision, built to run agentic tasks for hours. Supported on Inference Router and model synthesis to maximize intelligence per dollar. No setup, no model ops.

.Kimi.ai K3 from Moonshot AI is now live on DigitalOcean Inference Engine. 1M-token context, native vision, built to run agentic tasks for hours. Supported on Inference Router and model synthesis to maximize intelligence per dollar. No setup, no model ops.

DigitalOcean

667,886 views • 2 months ago

AkashML is live! A fully managed AI inference service on Akash Network. Deploying open models like DeepSeek shouldn’t cost $50K/month or require complex setup. AkashML makes inference simple, affordable, and automatic, so developers can run AI workloads on decentralized GPUs with ease. Try it out here 👇 Read More about it Here 👇

AkashML is live! A fully managed AI inference service on Akash Network. Deploying open models like DeepSeek shouldn’t cost $50K/month or require complex setup. AkashML makes inference simple, affordable, and automatic, so developers can run AI workloads on decentralized GPUs with ease. Try it out here 👇 Read More about it Here 👇

Akash Network

37,077 views • 10 months ago

$NVDA has the “lowest cost per token in the world” which is really a claim about owning the economics of inference. The company that delivers the most tokens per watt, per rack and per dollar is the one that captures the most demand and the best margins as AI scales.

$NVDA has the “lowest cost per token in the world” which is really a claim about owning the economics of inference. The company that delivers the most tokens per watt, per rack and per dollar is the one that captures the most demand and the best margins as AI scales.

Shay Boloor

48,594 views • 5 months ago

Walkthrough Google’s on-device genAI solutions with us — now available for experimentation. #AndroidAI We cover: what is on-device inference, new experimental access of the latest Gemini Nano model, sample apps, MediaPipe, and more. Full video →

Walkthrough Google’s on-device genAI solutions with us — now available for experimentation. #AndroidAI We cover: what is on-device inference, new experimental access of the latest Gemini Nano model, sample apps, MediaPipe, and more. Full video →

Android Developers

10,327 views • 2 years ago

Today we’re launching Model Vault — a dedicated, fully managed platform to run Cohere models securely and at scale. Model Vault delivers the control and isolation of self-hosting, without the operational burden: 🔒 Dedicated, isolated VPC ⚡ No noisy neighbors or rate limits 🔁 Elastic inference with predictable performance 📈 Real-time usage and performance monitoring No sharing. No limits. No surprises. Let Model Vault take care of model scaling and serving — so you can focus on building.

Today we’re launching Model Vault — a dedicated, fully managed platform to run Cohere models securely and at scale. Model Vault delivers the control and isolation of self-hosting, without the operational burden: 🔒 Dedicated, isolated VPC ⚡ No noisy neighbors or rate limits 🔁 Elastic inference with predictable performance 📈 Real-time usage and performance monitoring No sharing. No limits. No surprises. Let Model Vault take care of model scaling and serving — so you can focus on building.

Cohere

29,632 views • 8 months ago

🚨 Open-source AI just got serious. Alibaba just released Qwen 3.5-Plus 🤯 397B parameters. Only 17B active per inference. Performance comparable to Gemini 3 Pro — at just $0.8 per million tokens (1/18 the cost). But here’s what makes it different 👇 This isn’t just another benchmark drop. In the demo, Qwen runs directly inside the terminal: ➙ Reviews GitHub PRs ➙ Fetches diffs automatically ➙ Runs CLI & smoke tests ➙ Generates integration tests ➙ Executes /context ➙ Shows real-time token usage breakdown That’s not a chatbot. That’s a real developer workflow. Under the hood: ⚡ 60% lower deployment memory ⚡ Inference throughput increased by 8.6× (32K context) ⚡ Inference throughput increased by up to 19× (256K ultra-long context) ⚡ Native multimodal model ⚡ Supports direct video input (up to 2 hours) And the ecosystem is massive: ✔ 201 languages ✔ 250K vocabulary ✔ 400+ open-sourced models ✔ 1B+ downloads ✔ 200K+ derivative models Available now via Qwen App, MoDa Community, Hugging Face, and Alibaba Cloud Bailian. ✅ The AI race isn’t about who’s biggest anymore. → It’s about who’s usable. 🔥 Watch the demo 👇 #Qwen #OpenSourceAI #AI #DevTools

🚨 Open-source AI just got serious. Alibaba just released Qwen 3.5-Plus 🤯 397B parameters. Only 17B active per inference. Performance comparable to Gemini 3 Pro — at just $0.8 per million tokens (1/18 the cost). But here’s what makes it different 👇 This isn’t just another benchmark drop. In the demo, Qwen runs directly inside the terminal: ➙ Reviews GitHub PRs ➙ Fetches diffs automatically ➙ Runs CLI & smoke tests ➙ Generates integration tests ➙ Executes /context ➙ Shows real-time token usage breakdown That’s not a chatbot. That’s a real developer workflow. Under the hood: ⚡ 60% lower deployment memory ⚡ Inference throughput increased by 8.6× (32K context) ⚡ Inference throughput increased by up to 19× (256K ultra-long context) ⚡ Native multimodal model ⚡ Supports direct video input (up to 2 hours) And the ecosystem is massive: ✔ 201 languages ✔ 250K vocabulary ✔ 400+ open-sourced models ✔ 1B+ downloads ✔ 200K+ derivative models Available now via Qwen App, MoDa Community, Hugging Face, and Alibaba Cloud Bailian. ✅ The AI race isn’t about who’s biggest anymore. → It’s about who’s usable. 🔥 Watch the demo 👇 #Qwen #OpenSourceAI #AI #DevTools

Toha Khan

123,375 views • 7 months ago

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI Baseten Philip Kiely and ali explain what actually happens after a model is trained, why turning weights into a fast and reliable product creates an entirely new optimization problem, how quantization errors can cancel out to unlock more throughput, why inference teams are still finding 20–200% performance gains, how video generation runs into a quadratic attention wall, and how GLM-5.2 helped rewrite and optimize the GPU kernels serving GLM-5.2 itself.

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI Baseten Philip Kiely and ali explain what actually happens after a model is trained, why turning weights into a fast and reliable product creates an entirely new optimization problem, how quantization errors can cancel out to unlock more throughput, why inference teams are still finding 20–200% performance gains, how video generation runs into a quadratic attention wall, and how GLM-5.2 helped rewrite and optimize the GPU kernels serving GLM-5.2 itself.

Latent.Space

148,203 views • 1 month ago

✨ The #NVIDIARubin platform is engineered to master multi-step problem-solving and massive long-context reasoning workflows at scale. Built using extreme co-design, Rubin delivers more AI tokens per watt and lower cost per token compared to the previous architecture. ✔️ 50 PFLOPS of NVFP4 inference ✔️ 22 TB/s of HBM4 bandwidth ✔️ 3.6 TB/s of NVIDIA NVLink bandwidth per GPU Learn more about NVIDIA Rubin ➡️

✨ The #NVIDIARubin platform is engineered to master multi-step problem-solving and massive long-context reasoning workflows at scale. Built using extreme co-design, Rubin delivers more AI tokens per watt and lower cost per token compared to the previous architecture. ✔️ 50 PFLOPS of NVFP4 inference ✔️ 22 TB/s of HBM4 bandwidth ✔️ 3.6 TB/s of NVIDIA NVLink bandwidth per GPU Learn more about NVIDIA Rubin ➡️

NVIDIA AI Infrastructure

10,485 views • 8 months ago

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

TBPN

33,278 views • 6 months ago

Interview with Nebius Co-Founder Roman Chernin Please like & share this video so that all $NBIS investors on X will see it! :) If you prefer watching on YouTube: Timestamps: 00:00 - Why AI Infrastructure Is So Hard to Understand 00:24 - Market Fragmentation and What Actually Differentiates Providers 01:30 - Consolidation, Segmentation, and the Future AI Cloud Landscape 02:56 - What Analysts and VCs Still Get Wrong About AI Infrastructure 05:34 - Nebius Cloud: Product Readiness and Customer Proof Points 07:42 - Why Inference Workloads Are Exploding 09:11 - Training vs. Inference: How AI Models Actually Reach Production 10:10 - Why Inference Market Share May Concentrate Around a Few Winners 12:36 - Customer Use Cases: Coding, Enterprise AI, and Real-World Adoption 14:01 - Why Integrated Training and Inference Matter Strategically 16:01 - Building Scalable AI Infrastructure With High Utilization 18:24 - Token Factory: Inference as a Managed Service 20:24 - Revolut Case Study: AI-Driven Product Enhancements 22:56 - Token Factory Performance Optimization and Competitive Advantage 25:07 - Scale, Capacity, and Efficiency as Growth Drivers 28:36 - Why Inference Capacity Could Become the Next Major Bottleneck 30:10 - How Nebius Benchmarks Performance Across Providers 33:14 - The Future Size and Shape of the Inference Market 36:38 - Value-Based Pricing: Moving Beyond Cost per GPU Hour 40:55 - How Nebius Wins Deals: Quality, Performance, and Customer Experience 44:53 - Autonomous AI Platforms and the Rise of Agent-Based Models 47:28 - Tavily, Agentic Applications, and the Next Layer of the AI Stack 50:45 - Strategic Trade-Offs: Scaling, Product Roadmap, and Customer Relevance 55:40 - Final Thoughts: Adapting to the Next Shift in AI Workloads Nebius Roman Chernin

Interview with Nebius Co-Founder Roman Chernin Please like & share this video so that all $NBIS investors on X will see it! :) If you prefer watching on YouTube: Timestamps: 00:00 - Why AI Infrastructure Is So Hard to Understand 00:24 - Market Fragmentation and What Actually Differentiates Providers 01:30 - Consolidation, Segmentation, and the Future AI Cloud Landscape 02:56 - What Analysts and VCs Still Get Wrong About AI Infrastructure 05:34 - Nebius Cloud: Product Readiness and Customer Proof Points 07:42 - Why Inference Workloads Are Exploding 09:11 - Training vs. Inference: How AI Models Actually Reach Production 10:10 - Why Inference Market Share May Concentrate Around a Few Winners 12:36 - Customer Use Cases: Coding, Enterprise AI, and Real-World Adoption 14:01 - Why Integrated Training and Inference Matter Strategically 16:01 - Building Scalable AI Infrastructure With High Utilization 18:24 - Token Factory: Inference as a Managed Service 20:24 - Revolut Case Study: AI-Driven Product Enhancements 22:56 - Token Factory Performance Optimization and Competitive Advantage 25:07 - Scale, Capacity, and Efficiency as Growth Drivers 28:36 - Why Inference Capacity Could Become the Next Major Bottleneck 30:10 - How Nebius Benchmarks Performance Across Providers 33:14 - The Future Size and Shape of the Inference Market 36:38 - Value-Based Pricing: Moving Beyond Cost per GPU Hour 40:55 - How Nebius Wins Deals: Quality, Performance, and Customer Experience 44:53 - Autonomous AI Platforms and the Rise of Agent-Based Models 47:28 - Tavily, Agentic Applications, and the Next Layer of the AI Stack 50:45 - Strategic Trade-Offs: Scaling, Product Roadmap, and Customer Relevance 55:40 - Final Thoughts: Adapting to the Next Shift in AI Workloads Nebius Roman Chernin

Daniel Koss

204,842 views • 5 months ago

My full interview with Roman Chernin, Co-founder & Chief Business Officer of Nebius Nebius just signed a $17 billion deal with Microsoft and a $3 billion deal with Meta. These are two of the biggest tech companies on Earth - and they're coming to Nebius for AI infrastructure. But the company does so much more than that: - Full-stack AI cloud (data centers → software → managed services) - Nebius Token Factory Nebius Token Factory for managed inference & post-training - Partners include Shopify, Higgsfield, Jetbrains My 5 takeaways with Roman Chernin below - thanks for the conversation! Full interview also available on YT ⬇️ Timestamps: 00:00 - Introduction 01:47 - Overview of Nebius $NBIS 03:23 - Roman's background and role at Nebius 04:10 - The $17B Microsoft & $3B Meta deals 05:29 - Why hyperscalers trust Nebius over building in-house 07:06 - Nebius's full-stack approach: Data centers to managed services 08:39 - Customer segmentation: Hyperscalers & Frontier AI Labs vs AI Startups vs Enterprises 14:41 - Token Factory explained: Managed inference & post-training 17:41 - When to switch from closed-source to open-source models 20:59 - Real results: Process achieves 26% cost reduction 25:05 - Why developers love building on Nebius 28:40 - AI agents and the future of infrastructure interaction 32:22 - NVIDIA's Groq acquisition: What it means for inference 35:38 - Why AI adoption will surprise us

My full interview with Roman Chernin, Co-founder & Chief Business Officer of Nebius Nebius just signed a $17 billion deal with Microsoft and a $3 billion deal with Meta. These are two of the biggest tech companies on Earth - and they're coming to Nebius for AI infrastructure. But the company does so much more than that: - Full-stack AI cloud (data centers → software → managed services) - Nebius Token Factory Nebius Token Factory for managed inference & post-training - Partners include Shopify, Higgsfield, Jetbrains My 5 takeaways with Roman Chernin below - thanks for the conversation! Full interview also available on YT ⬇️ Timestamps: 00:00 - Introduction 01:47 - Overview of Nebius $NBIS 03:23 - Roman's background and role at Nebius 04:10 - The $17B Microsoft & $3B Meta deals 05:29 - Why hyperscalers trust Nebius over building in-house 07:06 - Nebius's full-stack approach: Data centers to managed services 08:39 - Customer segmentation: Hyperscalers & Frontier AI Labs vs AI Startups vs Enterprises 14:41 - Token Factory explained: Managed inference & post-training 17:41 - When to switch from closed-source to open-source models 20:59 - Real results: Process achieves 26% cost reduction 25:05 - Why developers love building on Nebius 28:40 - AI agents and the future of infrastructure interaction 32:22 - NVIDIA's Groq acquisition: What it means for inference 35:38 - Why AI adoption will surprise us

Elliot Garreffa

29,888 views • 7 months ago

The defining differentiator in AI right now isn't model performance, it's cost per token. Because in this market, efficiency is what turns demand into actual growth. Roman Chernin of Nebius outlines why inference economics are becoming the central challenge, as companies operating on thin margins need infrastructure that can scale without eroding profitability. Token Factory reflects that shift, focusing on orchestration, optimization, and full-stack control to make AI systems economically viable.

The defining differentiator in AI right now isn't model performance, it's cost per token. Because in this market, efficiency is what turns demand into actual growth. Roman Chernin of Nebius outlines why inference economics are becoming the central challenge, as companies operating on thin margins need infrastructure that can scale without eroding profitability. Token Factory reflects that shift, focusing on orchestration, optimization, and full-stack control to make AI systems economically viable.

Six Five Media

294,803 views • 5 months ago

New on DigitalOcean's Inference Engine: GLM-5.3-Flash from Z.ai. Agentic coding that rivals top-tier models, now natively multimodal for browser and computer use. 1M token context, MoE architecture, one endpoint swap to try it. Senior Developer Advocate Ryan O'Connor breaks it down, and Z.ai for Startups is there to help AI startups build faster with GLM models. 👇

New on DigitalOcean's Inference Engine: GLM-5.3-Flash from Z.ai. Agentic coding that rivals top-tier models, now natively multimodal for browser and computer use. 1M token context, MoE architecture, one endpoint swap to try it. Senior Developer Advocate Ryan O'Connor breaks it down, and Z.ai for Startups is there to help AI startups build faster with GLM models. 👇

DigitalOcean

140,726 views • 1 month ago

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!

Mike

102,223 views • 9 months ago

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."

TBPN

30,608 views • 4 months ago

Exciting update for ComfyUI users on Intel hardware! - The new OpenVINO custom node is now available for ComfyUI - Unlock faster AI inference on Intel CPUs, GPUs, and NPUs - No hardware upgrade needed, just install the ComfyUI-OpenVINO node Enjoy smoother, more efficient workflows with Intel acceleration 🧵 ↓

Exciting update for ComfyUI users on Intel hardware! - The new OpenVINO custom node is now available for ComfyUI - Unlock faster AI inference on Intel CPUs, GPUs, and NPUs - No hardware upgrade needed, just install the ComfyUI-OpenVINO node Enjoy smoother, more efficient workflows with Intel acceleration 🧵 ↓

ComfyUI

10,378 views • 1 year ago

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,622 views • 2 years ago

As AI labs race to train and deploy new frontier models, existing models become more affordable with better tokenomics. ✨ "Everybody's trying to get to the next frontier. And every time they get to the next frontier, the last generation AI tokens, the cost starts to decline about a factor of 10x every year," said NVIDIA CEO Jensen Huang in a recent keynote. Model optimization techniques such as speculative decoding and multi-token prediction, combined with inference serving platforms like NVIDIA Dynamo on NVIDIA Blackwell NVL72 systems, enable AI factories to boost throughput by 10x with one-tenth of the cost per token. Learn more about AI factory tokenomics ➡️

As AI labs race to train and deploy new frontier models, existing models become more affordable with better tokenomics. ✨ "Everybody's trying to get to the next frontier. And every time they get to the next frontier, the last generation AI tokens, the cost starts to decline about a factor of 10x every year," said NVIDIA CEO Jensen Huang in a recent keynote. Model optimization techniques such as speculative decoding and multi-token prediction, combined with inference serving platforms like NVIDIA Dynamo on NVIDIA Blackwell NVL72 systems, enable AI factories to boost throughput by 10x with one-tenth of the cost per token. Learn more about AI factory tokenomics ➡️

NVIDIA AI

16,053 views • 8 months ago

Moonshot AI limiting Kimi K3 subscriptions because of compute constraints is a massive signal because the 2.8 trillion parameter model is already trained and the bottleneck is now emerging as usage scales. Serving that demand requires a ton more accelerators, enormous memory capacity and faster networking to keep data moving efficiently across the cluster. So as AI spending shifts toward serving millions or billions of daily requests then memory and networking become critical bottlenecks giving $MU, $SKHY, $AVGO, $CRDO, $ALAB and $MRVL direct exposure to inference at scale.

Moonshot AI limiting Kimi K3 subscriptions because of compute constraints is a massive signal because the 2.8 trillion parameter model is already trained and the bottleneck is now emerging as usage scales. Serving that demand requires a ton more accelerators, enormous memory capacity and faster networking to keep data moving efficiently across the cluster. So as AI spending shifts toward serving millions or billions of daily requests then memory and networking become critical bottlenecks giving $MU, $SKHY, $AVGO, $CRDO, $ALAB and $MRVL direct exposure to inference at scale.

Shay Boloor

123,791 views • 2 months ago

Meta’s Segment Anything Model (SAM) 3.1 is now available on Meta Model API, giving developers a fast and lightweight model for detection, segmentation and tracking in a single call on inference tuned for SAM 3.1's architecture. Use a short phrase to find objects in images and video. One API call returns detections, pixel-precise segmentation masks, and identity-preserving video tracks. Learn more and start building:

Meta’s Segment Anything Model (SAM) 3.1 is now available on Meta Model API, giving developers a fast and lightweight model for detection, segmentation and tracking in a single call on inference tuned for SAM 3.1's architecture. Use a short phrase to find objects in images and video. One API call returns detections, pixel-precise segmentation masks, and identity-preserving video tracks. Learn more and start building:

Meta for Developers

567,545 views • 12 days ago