正在加载视频...

视频加载失败

返回首页

70% lower cost. 84% fewer GPT calls. 90% expert-preferred output. At DevCon 6, Dr. David Zihr, Medical Director for Physician Advising Services at Tampa General Hospital, and Palantir Forward Deployed Engineer Colton Rusch show how AIP Evolve achieved these optimizations by screening models across providers, rewriting prompts, and swapping...

PalantirTech's profile picture

Palantir

443,391 subscribers

73,516 次观看 • 1 个月前 •via X (Twitter)

科学技术健康养生新闻政治
LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

See how Palantir Software Engineer Colton Rusch uses AIP Evolve to prove that cheaper AI doesn't have to mean worse AI: "Evolve screened a whole suite of models across providers, then used evals and expert feedback to rule out certain options and keep others for further optimization." "We got to the point where evolve produced a vastly cheaper version — 68% lower compute costs." "We got lower costs and better quality at the same time."
2:53

See how Palantir Software Engineer Colton Rusch uses AIP Evolve to prove that cheaper AI doesn't have to mean worse AI: "Evolve screened a whole suite of models across providers, then used evals and expert feedback to rule out certain options and keep others for further optimization." "We got to the point where evolve produced a vastly cheaper version — 68% lower compute costs." "We got lower costs and better quality at the same time."

Palantir

13,673 次观看 • 1 个月前

Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.
1:05

Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.

Milk Road AI

27,908 次观看 • 1 个月前

How is Arbaaz, CEO of making AI models that continue to learn from your business data and continue to grow? He is working with car dealerships now, but growing to other businesses soon. Here is what Grok says you will learn from this video: +++++ By watching this podcast episode, viewers will gain insights into how AI is being practically applied in business, particularly in niche industries like car dealerships, while also exploring broader AI concepts, challenges, and future implications. Here's a breakdown of the main takeaways: AI Customization for Businesses: Learn how Polycom Computing builds specialized AI models that continuously train on a company's real-time data and workflows, acting as "companions" rather than generic tools. This contrasts with foundational models from companies like OpenAI or Anthropic, which struggle to adapt to specific "worlds" without losing efficiency. The focus is on personalization to avoid wasting attention, intelligence, and money. Pivoting AI Strategies for Revenue Growth: Understand the shift from cost-cutting (e.g., automating call centers) to revenue-increasing applications. Urba explains why targeting high-value tasks like sales in car dealerships creates defensible moats, as opposed to commoditized cost reductions. This includes automating complex funnels—from lead submission to financing—while ensuring compliance with regulations and seamless integration with existing teams. Scalable AI Agents in Practice: Discover how AI agents must learn autonomously (e.g., adapting to different CRMs, processes, and preferences across dealerships) to avoid becoming non-scalable consulting services. Key challenges include creating "glue" between humans and AI, avoiding hard-coded rules, and using web actions to integrate siloed software like Dealer Management Systems (DMS). Boosting Sales with AI Techniques: Gain knowledge on tactics like rapid response (replying within 5 minutes boosts conversion 22x), creating natural "disfluencies" (typos, emojis, jokes) for human-like communication, managing after-hours leads, and educating customers without pushing sales. Pilots showed 50% sales increases by handling unanswered leads (70% go ignored), qualifying buyers, and maintaining conversation threads. Multi-Agent Systems and Proactive AI: Explore the difference between reactive Q&A models and proactive agents with agency—they predict, act, and update based on goals. Building these systems reveals bottlenecks (e.g., overwhelming businesses with leads), leading to solutions like AI buying cars or linking sales/service arms. Urba discusses how AI plateaus without personalization, risking model collapse or equilibrium where gains cancel out. Technical Deep Dives into AI Development: Get explanations of advanced concepts like real-time RLHF (Reinforcement Learning from Human Feedback) for continuous improvement, coherence (maintaining logical consistency across outputs), dynamic tokenization for new abstractions, and evaluating models (e.g., avoiding memorization over reasoning, energy limits in scaling). Viewers see a demo of their platform for tasks like quant strategies, presidential analysis, and training runs. Selling and Adopting AI in Traditional Industries: Learn how to pitch AI to non-tech audiences (e.g., car dealers) by focusing on results—more money, fewer bottlenecks, centralized dashboards—rather than jargon. Emphasize empathy: make owners feel smart, reduce reliance on salespeople, and return control via AI-managed customer databases. Broader AI Implications and Misconceptions: Understand why AI won't create a "machine god" that eliminates all jobs—it's bound by physics (e.g., energy needs), expands economies (like the Industrial Revolution), and requires human perspectives for true advantage. AI enhances productivity, creates new roles, and democratizes power, but risks arise from misuse, not inherent agency. Urba stresses proactive defense through widespread AI proficiency. Overall, the episode bridges entrepreneurial stories, technical AI mechanics, and real-world applications, making it valuable for entrepreneurs, AI enthusiasts, and business owners curious about integrating AI without hype. It's a candid look at building scalable, impactful AI beyond buzzwords.
57:31

How is Arbaaz, CEO of making AI models that continue to learn from your business data and continue to grow? He is working with car dealerships now, but growing to other businesses soon. Here is what Grok says you will learn from this video: +++++ By watching this podcast episode, viewers will gain insights into how AI is being practically applied in business, particularly in niche industries like car dealerships, while also exploring broader AI concepts, challenges, and future implications. Here's a breakdown of the main takeaways: AI Customization for Businesses: Learn how Polycom Computing builds specialized AI models that continuously train on a company's real-time data and workflows, acting as "companions" rather than generic tools. This contrasts with foundational models from companies like OpenAI or Anthropic, which struggle to adapt to specific "worlds" without losing efficiency. The focus is on personalization to avoid wasting attention, intelligence, and money. Pivoting AI Strategies for Revenue Growth: Understand the shift from cost-cutting (e.g., automating call centers) to revenue-increasing applications. Urba explains why targeting high-value tasks like sales in car dealerships creates defensible moats, as opposed to commoditized cost reductions. This includes automating complex funnels—from lead submission to financing—while ensuring compliance with regulations and seamless integration with existing teams. Scalable AI Agents in Practice: Discover how AI agents must learn autonomously (e.g., adapting to different CRMs, processes, and preferences across dealerships) to avoid becoming non-scalable consulting services. Key challenges include creating "glue" between humans and AI, avoiding hard-coded rules, and using web actions to integrate siloed software like Dealer Management Systems (DMS). Boosting Sales with AI Techniques: Gain knowledge on tactics like rapid response (replying within 5 minutes boosts conversion 22x), creating natural "disfluencies" (typos, emojis, jokes) for human-like communication, managing after-hours leads, and educating customers without pushing sales. Pilots showed 50% sales increases by handling unanswered leads (70% go ignored), qualifying buyers, and maintaining conversation threads. Multi-Agent Systems and Proactive AI: Explore the difference between reactive Q&A models and proactive agents with agency—they predict, act, and update based on goals. Building these systems reveals bottlenecks (e.g., overwhelming businesses with leads), leading to solutions like AI buying cars or linking sales/service arms. Urba discusses how AI plateaus without personalization, risking model collapse or equilibrium where gains cancel out. Technical Deep Dives into AI Development: Get explanations of advanced concepts like real-time RLHF (Reinforcement Learning from Human Feedback) for continuous improvement, coherence (maintaining logical consistency across outputs), dynamic tokenization for new abstractions, and evaluating models (e.g., avoiding memorization over reasoning, energy limits in scaling). Viewers see a demo of their platform for tasks like quant strategies, presidential analysis, and training runs. Selling and Adopting AI in Traditional Industries: Learn how to pitch AI to non-tech audiences (e.g., car dealers) by focusing on results—more money, fewer bottlenecks, centralized dashboards—rather than jargon. Emphasize empathy: make owners feel smart, reduce reliance on salespeople, and return control via AI-managed customer databases. Broader AI Implications and Misconceptions: Understand why AI won't create a "machine god" that eliminates all jobs—it's bound by physics (e.g., energy needs), expands economies (like the Industrial Revolution), and requires human perspectives for true advantage. AI enhances productivity, creates new roles, and democratizes power, but risks arise from misuse, not inherent agency. Urba stresses proactive defense through widespread AI proficiency. Overall, the episode bridges entrepreneurial stories, technical AI mechanics, and real-world applications, making it valuable for entrepreneurs, AI enthusiasts, and business owners curious about integrating AI without hype. It's a candid look at building scalable, impactful AI beyond buzzwords.

Robert Scoble

59,714 次观看 • 1 年前

Elon Musk just pulled off the biggest AI power grab of 2026. Tesla is capping every employee at $200 a week on AI spending starting Monday, July 6. Media's celebrating it as cost control. But what Elon actually built is an expense policy that redirects his own engineering workforce off Claude and onto Grok, while every competitor gets throttled by internal procurement rules. Here's what happened: Tesla spent the last six months pushing engineers to use AI as aggressively as possible. Leadership built an internal platform called Bottle Rocket that gave employees access to Claude, GPT, Gemini, Grok, and Cursor. They gamified adoption by ranking engineers on internal leaderboards by how many AI tokens they consumed. The strategy worked. Software engineers started burning THOUSANDS of dollars a week on Claude and Cursor. Then the invoices arrived and Tesla panicked. But they didn't pull the standard cost-control response... The loophole: The $200 weekly cap does not apply to beta products from xAI. Grok is completely exempt from the cap. Anthropic's Claude, OpenAI's GPT, and Google's Gemini all get throttled at the same $200 line. Four Tesla engineers told Electrek that internal usage overwhelmingly favors Claude over Grok. That preference is about to become financially punishing overnight. The genius part: This quarter SpaceX is closing a $60 billion all-stock acquisition of Anysphere, the parent company of Cursor. The moment that deal closes, Cursor's Composer coding model falls under the same Musk-controlled ecosystem, and any Tesla engineer choosing between a capped Claude session and an uncapped Composer session will pay a financial penalty for using the tool they actually prefer. By exempting only his own products from the cap, Elon is using Tesla shareholder money to build market share for xAI without ever having to disclose that is what he is doing. Because on paper, it is cost control. Now zoom out to what this signals for the wider AI narrative: Uber capped employees at $1,500 a month after burning $3.4 billion in four months. Meta introduced spending caps. Amazon and Walmart pushed staff toward cheaper models. Microsoft canceled Claude Code licenses across 100,000 engineers. Every Fortune 500 that pushed heavy AI adoption in 2025 is now rationing it in 2026. Meanwhile Nvidia is trading at a $5 trillion market cap. That entire valuation assumes enterprise AI consumption is about to explode across the economy. But every company actually deploying AI at scale is telling their own engineers to slow down. One of these narratives is lying. Goldman Sachs still forecasts a 24x increase in token consumption by 2030. Gartner says total enterprise AI costs will keep climbing because agents consume exponentially more tokens per task. Jensen Huang keeps repeating that 100 AI agents will work alongside every employee. And now the CEO of the most agentic company on the planet just told his own engineers they cannot spend more than $200 a week on the tools those agents need to run. Retail investors buying Nvidia and Palantir today are betting enterprise AI adoption compounds without limit. The CEOs deploying AI inside those same enterprises are betting the exact opposite, in writing, by internal memo. Thoughts?
1:18

Elon Musk just pulled off the biggest AI power grab of 2026. Tesla is capping every employee at $200 a week on AI spending starting Monday, July 6. Media's celebrating it as cost control. But what Elon actually built is an expense policy that redirects his own engineering workforce off Claude and onto Grok, while every competitor gets throttled by internal procurement rules. Here's what happened: Tesla spent the last six months pushing engineers to use AI as aggressively as possible. Leadership built an internal platform called Bottle Rocket that gave employees access to Claude, GPT, Gemini, Grok, and Cursor. They gamified adoption by ranking engineers on internal leaderboards by how many AI tokens they consumed. The strategy worked. Software engineers started burning THOUSANDS of dollars a week on Claude and Cursor. Then the invoices arrived and Tesla panicked. But they didn't pull the standard cost-control response... The loophole: The $200 weekly cap does not apply to beta products from xAI. Grok is completely exempt from the cap. Anthropic's Claude, OpenAI's GPT, and Google's Gemini all get throttled at the same $200 line. Four Tesla engineers told Electrek that internal usage overwhelmingly favors Claude over Grok. That preference is about to become financially punishing overnight. The genius part: This quarter SpaceX is closing a $60 billion all-stock acquisition of Anysphere, the parent company of Cursor. The moment that deal closes, Cursor's Composer coding model falls under the same Musk-controlled ecosystem, and any Tesla engineer choosing between a capped Claude session and an uncapped Composer session will pay a financial penalty for using the tool they actually prefer. By exempting only his own products from the cap, Elon is using Tesla shareholder money to build market share for xAI without ever having to disclose that is what he is doing. Because on paper, it is cost control. Now zoom out to what this signals for the wider AI narrative: Uber capped employees at $1,500 a month after burning $3.4 billion in four months. Meta introduced spending caps. Amazon and Walmart pushed staff toward cheaper models. Microsoft canceled Claude Code licenses across 100,000 engineers. Every Fortune 500 that pushed heavy AI adoption in 2025 is now rationing it in 2026. Meanwhile Nvidia is trading at a $5 trillion market cap. That entire valuation assumes enterprise AI consumption is about to explode across the economy. But every company actually deploying AI at scale is telling their own engineers to slow down. One of these narratives is lying. Goldman Sachs still forecasts a 24x increase in token consumption by 2030. Gartner says total enterprise AI costs will keep climbing because agents consume exponentially more tokens per task. Jensen Huang keeps repeating that 100 AI agents will work alongside every employee. And now the CEO of the most agentic company on the planet just told his own engineers they cannot spend more than $200 a week on the tools those agents need to run. Retail investors buying Nvidia and Palantir today are betting enterprise AI adoption compounds without limit. The CEOs deploying AI inside those same enterprises are betting the exact opposite, in writing, by internal memo. Thoughts?

Ricardo

188,797 次观看 • 1 个月前

Microsoft just banned its own engineers from using AI. The tool was literally costing MORE than the humans it was supposed to replace. They lied to you about AI adoption and now the whole narrative is blowing up: Microsoft gave thousands of engineers access to Claude Code six months ago and encouraged them to use it. Engineers loved it and adoption exploded. But then the invoices arrived. Token-based pricing means every query, every code review, every debugging session costs money. At scale across 100,000 engineers, the numbers became so large that Microsoft issued an internal order to cancel nearly all Claude Code licenses by end of June and force everyone onto their own cheaper tool instead. The company that invested $5 billion in Anthropic just told its own people to stop using Anthropic's product because it costs too much. Uber's story is even worse... Their CTO Praveen Neppalli Naga told The Information that the budget he planned for the full year was "blown away already" by April. Uber had rolled out Claude Code in December 2025. By March, 84% of their 5,000 engineers were using it with 70% of all committed code coming from AI systems. Heavy users were burning $500 to $2,000 per month each. Naga himself spent $1,200 in a single two-hour demo session. The company had even built internal leaderboards ranking engineers by how much AI they used. They literally gamified the spending and then ran out of money. Now look at what Nvidia's own VP of applied deep learning Bryan Catanzaro said to Axios last month. Direct quote: "For my team, the cost of compute is far beyond the costs of the employees." This is a VP at the company that SELLS the chips saying that using AI is more expensive than paying humans. Think about what this means for the entire AI narrative. Every CEO on every earnings call for the past two years has said the same thing: AI will make us more efficient, reduce headcount, and cut costs. The stock market rewarded every company that said it. Fired workers, stock goes up. Announced AI adoption, stock goes up. But the actual companies deploying AI at scale are discovering the math doesn't work. The MORE employees use AI, the HIGHER the bill. Goldman Sachs forecasts a 24x increase in token consumption by 2030 as companies adopt AI agents. Gartner just published a report showing that even though individual token prices will drop 90% by 2030, total enterprise AI costs will go UP because agents consume exponentially more tokens per task than basic tools. Meta built an internal dashboard called "Claudeonomics" to track which employees use the most AI. Amazon started pushing engineers to "tokenmaxx," their internal term for consuming as many AI tokens as possible. Both companies are spending hundreds of billions on AI infrastructure this year alone. And Microsoft, the company that bet its entire future on AI, just told 100,000 engineers to stop using the tool they liked best because the per-token bills got out of control. The companies building AI are telling investors it saves money. The companies using AI are finding out it costs more than the humans it was supposed to replace. And even the company that makes the chips just admitted it through its own VP. This is the gap nobody on Wall Street is pricing in. $725 billion in AI infrastructure spending this year across Big Tech. And the first companies to actually deploy these tools at scale are already pulling back because the economics don't work. What do you think?
1:33

Microsoft just banned its own engineers from using AI. The tool was literally costing MORE than the humans it was supposed to replace. They lied to you about AI adoption and now the whole narrative is blowing up: Microsoft gave thousands of engineers access to Claude Code six months ago and encouraged them to use it. Engineers loved it and adoption exploded. But then the invoices arrived. Token-based pricing means every query, every code review, every debugging session costs money. At scale across 100,000 engineers, the numbers became so large that Microsoft issued an internal order to cancel nearly all Claude Code licenses by end of June and force everyone onto their own cheaper tool instead. The company that invested $5 billion in Anthropic just told its own people to stop using Anthropic's product because it costs too much. Uber's story is even worse... Their CTO Praveen Neppalli Naga told The Information that the budget he planned for the full year was "blown away already" by April. Uber had rolled out Claude Code in December 2025. By March, 84% of their 5,000 engineers were using it with 70% of all committed code coming from AI systems. Heavy users were burning $500 to $2,000 per month each. Naga himself spent $1,200 in a single two-hour demo session. The company had even built internal leaderboards ranking engineers by how much AI they used. They literally gamified the spending and then ran out of money. Now look at what Nvidia's own VP of applied deep learning Bryan Catanzaro said to Axios last month. Direct quote: "For my team, the cost of compute is far beyond the costs of the employees." This is a VP at the company that SELLS the chips saying that using AI is more expensive than paying humans. Think about what this means for the entire AI narrative. Every CEO on every earnings call for the past two years has said the same thing: AI will make us more efficient, reduce headcount, and cut costs. The stock market rewarded every company that said it. Fired workers, stock goes up. Announced AI adoption, stock goes up. But the actual companies deploying AI at scale are discovering the math doesn't work. The MORE employees use AI, the HIGHER the bill. Goldman Sachs forecasts a 24x increase in token consumption by 2030 as companies adopt AI agents. Gartner just published a report showing that even though individual token prices will drop 90% by 2030, total enterprise AI costs will go UP because agents consume exponentially more tokens per task than basic tools. Meta built an internal dashboard called "Claudeonomics" to track which employees use the most AI. Amazon started pushing engineers to "tokenmaxx," their internal term for consuming as many AI tokens as possible. Both companies are spending hundreds of billions on AI infrastructure this year alone. And Microsoft, the company that bet its entire future on AI, just told 100,000 engineers to stop using the tool they liked best because the per-token bills got out of control. The companies building AI are telling investors it saves money. The companies using AI are finding out it costs more than the humans it was supposed to replace. And even the company that makes the chips just admitted it through its own VP. This is the gap nobody on Wall Street is pricing in. $725 billion in AI infrastructure spending this year across Big Tech. And the first companies to actually deploy these tools at scale are already pulling back because the economics don't work. What do you think?

Ricardo

2,971,575 次观看 • 3 个月前

Sam Altman just dropped the most important interview of 2025. And buried in it are four numbers that explain why everything you think about AI is wrong. Here's what he revealed: Number 1: AI companies are generating 10 TRILLION tokens per day. Humans? Average 20,000 tokens per day. Sam's exact words: "Models will output more tokens than all of humanity put together. Then 10x that. Then 100x that." We're not talking about AI assisting human work anymore. We're talking about AI replacing the entire volume of human intellectual output on the planet. And most people have no idea this shift already happened. Number 2: OpenAI's enterprise business is CRUSHING consumer. Everyone thinks OpenAI is ChatGPT for normies. Wrong. Sam just revealed: "Enterprise growth OUTPACED consumer growth this year." The API business is growing faster than ChatGPT. Over 1 million enterprise users already. "If we had double the compute, we'd be at double the revenue right now." Translation: OpenAI isn't compute-constrained by technology. They're revenue-constrained by infrastructure. The bottleneck is supply and not demand. Every dollar of compute they add prints money. Number 3: GPT-5.2 beats you at 74% of your job. Sam revealed OpenAI's internal GDP-Val benchmark. It measures how AI performs on knowledge work tasks across 40+ verticals. The results: GPT-5.2 beats or ties expert-level knowledge workers at 74.1% of tasks. Legal analysis. PowerPoint decks. Web apps. Financial modeling. Customer support. Sam's description: "A co-worker you can assign an hour's worth of tasks to and get something you prefer back 3 out of 4 times." Three years ago, ChatGPT launched at basically 0% on this scale. Now it's at 74%. And that's not GPT-6. That's what's available RIGHT NOW. Most companies haven't even started using this yet. But here's what Sam said about the gap between capability and adoption: "The overhang is going to be massive. Most people are still asking similar questions they did in the GPT-4 realm." Translation: The models can do 10x more than people have figured out how to use them for. Which means there's a HUGE arbitrage opportunity. Early adopters who actually integrate this into workflows will dominate their industries before competitors even understand what happened. Number 4: AGI already happened. And nobody noticed. Sam's exact quote: "AGI kind of went whooshing by. We're in this fuzzy period where some people think we have it and some don't." Read that again. The CEO of OpenAI just said AGI might have already arrived and we're arguing about definitions while it's actively replacing knowledge work. He even moved the goalposts. The new benchmark: "Superintelligence" = when AI can be a better president or CEO than any human. Not "as good as." BETTER than. We went from "can AI pass a Turing test" to "can AI run countries better than humans" in 3 years. So what does this actually mean? The AI revolution isn't about chatbots getting smarter. It's about the complete replacement of human intellectual output with machine output. At scale. Across every industry. Faster than anyone's prepared for. And the companies positioning for this RIGHT NOW are the ones printing money. OpenAI's enterprise growth is outpacing consumer because businesses see what's coming. They're not buying "AI tools." They're buying the ability to 10x output without 10x-ing headcount. Sam said they'll triple their compute next year. Then triple it again. Revenue is growing even faster than that. "We have never found a situation where we can't monetize all the compute we have." If he isn't lying then that's literally a printing press. The market still doesn't get it. Everyone's focused on "AI bubble" fears while OpenAI is solving the only problem that matters: turning compute into revenue at a faster rate than they're spending. They're not hoping demand catches up to supply. Demand is already 2x ahead of what they can deliver. Meanwhile, most knowledge workers are still using GPT-4 prompts on GPT-5.2. The capability overhang is massive. The arbitrage window is open. And it's closing fast. If you're running a B2B business and you're not integrating AI at the level Sam just described, you're not "waiting to see how it plays out." You're getting crushed by competitors who already figured it out. The companies that win in 2026 won't be the ones with the best AI. They'll be the ones who understood what Sam just laid out 6 months before everyone else did.
1:33

Sam Altman just dropped the most important interview of 2025. And buried in it are four numbers that explain why everything you think about AI is wrong. Here's what he revealed: Number 1: AI companies are generating 10 TRILLION tokens per day. Humans? Average 20,000 tokens per day. Sam's exact words: "Models will output more tokens than all of humanity put together. Then 10x that. Then 100x that." We're not talking about AI assisting human work anymore. We're talking about AI replacing the entire volume of human intellectual output on the planet. And most people have no idea this shift already happened. Number 2: OpenAI's enterprise business is CRUSHING consumer. Everyone thinks OpenAI is ChatGPT for normies. Wrong. Sam just revealed: "Enterprise growth OUTPACED consumer growth this year." The API business is growing faster than ChatGPT. Over 1 million enterprise users already. "If we had double the compute, we'd be at double the revenue right now." Translation: OpenAI isn't compute-constrained by technology. They're revenue-constrained by infrastructure. The bottleneck is supply and not demand. Every dollar of compute they add prints money. Number 3: GPT-5.2 beats you at 74% of your job. Sam revealed OpenAI's internal GDP-Val benchmark. It measures how AI performs on knowledge work tasks across 40+ verticals. The results: GPT-5.2 beats or ties expert-level knowledge workers at 74.1% of tasks. Legal analysis. PowerPoint decks. Web apps. Financial modeling. Customer support. Sam's description: "A co-worker you can assign an hour's worth of tasks to and get something you prefer back 3 out of 4 times." Three years ago, ChatGPT launched at basically 0% on this scale. Now it's at 74%. And that's not GPT-6. That's what's available RIGHT NOW. Most companies haven't even started using this yet. But here's what Sam said about the gap between capability and adoption: "The overhang is going to be massive. Most people are still asking similar questions they did in the GPT-4 realm." Translation: The models can do 10x more than people have figured out how to use them for. Which means there's a HUGE arbitrage opportunity. Early adopters who actually integrate this into workflows will dominate their industries before competitors even understand what happened. Number 4: AGI already happened. And nobody noticed. Sam's exact quote: "AGI kind of went whooshing by. We're in this fuzzy period where some people think we have it and some don't." Read that again. The CEO of OpenAI just said AGI might have already arrived and we're arguing about definitions while it's actively replacing knowledge work. He even moved the goalposts. The new benchmark: "Superintelligence" = when AI can be a better president or CEO than any human. Not "as good as." BETTER than. We went from "can AI pass a Turing test" to "can AI run countries better than humans" in 3 years. So what does this actually mean? The AI revolution isn't about chatbots getting smarter. It's about the complete replacement of human intellectual output with machine output. At scale. Across every industry. Faster than anyone's prepared for. And the companies positioning for this RIGHT NOW are the ones printing money. OpenAI's enterprise growth is outpacing consumer because businesses see what's coming. They're not buying "AI tools." They're buying the ability to 10x output without 10x-ing headcount. Sam said they'll triple their compute next year. Then triple it again. Revenue is growing even faster than that. "We have never found a situation where we can't monetize all the compute we have." If he isn't lying then that's literally a printing press. The market still doesn't get it. Everyone's focused on "AI bubble" fears while OpenAI is solving the only problem that matters: turning compute into revenue at a faster rate than they're spending. They're not hoping demand catches up to supply. Demand is already 2x ahead of what they can deliver. Meanwhile, most knowledge workers are still using GPT-4 prompts on GPT-5.2. The capability overhang is massive. The arbitrage window is open. And it's closing fast. If you're running a B2B business and you're not integrating AI at the level Sam just described, you're not "waiting to see how it plays out." You're getting crushed by competitors who already figured it out. The companies that win in 2026 won't be the ones with the best AI. They'll be the ones who understood what Sam just laid out 6 months before everyone else did.

Ricardo

358,728 次观看 • 8 个月前

China just made Silicon Valley's entire AI industry look like a scam. The US government spent 3 years trying to stop China from building competitive AI. But this backfired HORRIBLY. Here's what happened: Yesterday, a Chinese startup called DeepSeek released a new AI model called V4. It matches the performance of OpenAI and Anthropic's best models. At 1/7th the price. And for the first time ever, it was built on Chinese chips. NOT American ones. That last part is the one that terrifies the west. For context: Since 2022, the US has banned the export of advanced AI chips to China. The entire strategy was built on the assumption that if China can't access Nvidia's best hardware, they can't build frontier AI. But DeepSeek just proved that assumption wrong. Their V4 model was trained and runs on Huawei's Ascend chips. Huawei spent months working directly with DeepSeek to make sure V4 runs across their entire line of AI processors. Jensen Huang even predicted this on a recent podcast: "The day that DeepSeek comes out on Huawei first, that is a horrible outcome for our nation." That day was yesterday. And the numbers are crazy: DeepSeek V4 costs $3.48 per million output tokens. OpenAI's latest model GPT-5.5 costs $30. Anthropic's Claude charges $25. Same ballpark performance. 7x cheaper. Uber's CTO just admitted they burned through their ENTIRE 2026 AI budget in 4 months using Anthropic's tools. If Uber had used DeepSeek instead, that same budget would have lasted 7 YEARS. 4 months vs 7 years. Same work getting done. But the pricing isn't even the big thing here. The real story is what DeepSeek did with their technical report: They published the benchmarks where they LOSE. Every AI company cherry-picks the tests where their model wins. DeepSeek ran the full comparison against GPT-5.4 and Google's Gemini, found they trail frontier models by 3 to 6 months, and printed it anyway. They literally don't care because the price gap makes the performance gap irrelevant for 90% of use cases. So the US export controls didn't slow China down. They ACCELERATED China's independence. Because Chinese developers were FORCED to train models with limited resources, they had to figure out how to make AI radically more efficient. That constraint became their competitive advantage. Every generation of DeepSeek has gotten dramatically cheaper to train. V4 continues the trend. Meanwhile US companies are going the OPPOSITE direction: OpenAI's GPT-5.5 Pro costs $180 per million output tokens. That's 51x more expensive than DeepSeek V4 for comparable work. The Commerce Secretary confirmed this week that ZERO Nvidia advanced chip shipments have actually gone through to China despite being approved in January. So China built frontier AI anyway. Without American chips. At a fraction of the cost. And the market response tells you everything: Chinese chipmaker SMIC surged 10%. Huahong Semiconductor jumped 15%. DeepSeek's Chinese AI competitors Zhipu AI and MiniMax dropped 9% because V4 is destroying them too. DeepSeek is making Silicon Valley's pricing model look like a scam. US tech companies spent $650 billion on AI infrastructure this year. DeepSeek just showed the world you can match their output for pennies. The export controls were supposed to be America's ace card. Instead they taught China how to win without American chips, at American prices nobody can compete with. Jensen Huang was right. This is a horrible outcome. But it's the outcome America built for itself.
1:06

China just made Silicon Valley's entire AI industry look like a scam. The US government spent 3 years trying to stop China from building competitive AI. But this backfired HORRIBLY. Here's what happened: Yesterday, a Chinese startup called DeepSeek released a new AI model called V4. It matches the performance of OpenAI and Anthropic's best models. At 1/7th the price. And for the first time ever, it was built on Chinese chips. NOT American ones. That last part is the one that terrifies the west. For context: Since 2022, the US has banned the export of advanced AI chips to China. The entire strategy was built on the assumption that if China can't access Nvidia's best hardware, they can't build frontier AI. But DeepSeek just proved that assumption wrong. Their V4 model was trained and runs on Huawei's Ascend chips. Huawei spent months working directly with DeepSeek to make sure V4 runs across their entire line of AI processors. Jensen Huang even predicted this on a recent podcast: "The day that DeepSeek comes out on Huawei first, that is a horrible outcome for our nation." That day was yesterday. And the numbers are crazy: DeepSeek V4 costs $3.48 per million output tokens. OpenAI's latest model GPT-5.5 costs $30. Anthropic's Claude charges $25. Same ballpark performance. 7x cheaper. Uber's CTO just admitted they burned through their ENTIRE 2026 AI budget in 4 months using Anthropic's tools. If Uber had used DeepSeek instead, that same budget would have lasted 7 YEARS. 4 months vs 7 years. Same work getting done. But the pricing isn't even the big thing here. The real story is what DeepSeek did with their technical report: They published the benchmarks where they LOSE. Every AI company cherry-picks the tests where their model wins. DeepSeek ran the full comparison against GPT-5.4 and Google's Gemini, found they trail frontier models by 3 to 6 months, and printed it anyway. They literally don't care because the price gap makes the performance gap irrelevant for 90% of use cases. So the US export controls didn't slow China down. They ACCELERATED China's independence. Because Chinese developers were FORCED to train models with limited resources, they had to figure out how to make AI radically more efficient. That constraint became their competitive advantage. Every generation of DeepSeek has gotten dramatically cheaper to train. V4 continues the trend. Meanwhile US companies are going the OPPOSITE direction: OpenAI's GPT-5.5 Pro costs $180 per million output tokens. That's 51x more expensive than DeepSeek V4 for comparable work. The Commerce Secretary confirmed this week that ZERO Nvidia advanced chip shipments have actually gone through to China despite being approved in January. So China built frontier AI anyway. Without American chips. At a fraction of the cost. And the market response tells you everything: Chinese chipmaker SMIC surged 10%. Huahong Semiconductor jumped 15%. DeepSeek's Chinese AI competitors Zhipu AI and MiniMax dropped 9% because V4 is destroying them too. DeepSeek is making Silicon Valley's pricing model look like a scam. US tech companies spent $650 billion on AI infrastructure this year. DeepSeek just showed the world you can match their output for pennies. The export controls were supposed to be America's ace card. Instead they taught China how to win without American chips, at American prices nobody can compete with. Jensen Huang was right. This is a horrible outcome. But it's the outcome America built for itself.

Ricardo

281,024 次观看 • 4 个月前

The U.S. MUST win the AI race We’ve implemented a clear policy at micro1: we will only work with U.S. AI labs and its allies. We made this decision because the AI race is not just about better products. It is about who controls the intelligence layer of the global economy, and whether frontier capability is used to strengthen the free world or to empower adversarial states. AI will be the most important technology of our lifetime. In the fullness of time, it will automate most functions across the economy. Not just software tasks, but coordination, production, logistics, judgment, and execution. As those functions are automated, human time is freed up to invent new ones. Those new functions then become candidates for automation themselves. This loop compounds. As this trajectory continues, output per worker increases dramatically. Entire categories of work become cheaper and faster to perform. Manufacturing reshoring becomes economically viable not because of policy intervention, but because intelligent systems operated domestically outperform global labor arbitrage. Goods and services trend toward lower marginal cost, while distribution improves through better coordination of supply and demand. That is the upside. However, this is impossible without deep integration of intelligent systems. For AI to meaningfully automate real-world functions inside enterprises or governments, it needs full context of any given enterprise. That means read and write access to its core databases. There is no credible path to automating high-impact functions without granting frontier systems that level of access. If the United States does not win the AI race, enterprises eventually face a constrained choice. Either grant that access to Chinese models controlled by an adversarial government, or rely on sub-optimal intelligence to automate functions that still must be automated. Both outcomes are not acceptable. And ultimately, this becomes the greatest national security risk the United States has ever faced. AI models are trained by humans. The judgment embedded in pre-training data and especially in expert post-training data largely determines how a model behaves. While emergent behavior exists, a useful approximation is that a model reflects the weighted aggregate of the human judgment distilled into it. Assisting foreign actors—who will naturally prioritize expert tasks aligned with their own interests—to dominate data creation embeds those interests directly into the intelligence layer itself. Once encoded at scale, these interests propagate through every downstream applications that relies on that intelligence. Here’s how we win. First, leverage is in software. China is ahead in hardware for physically intelligent systems. Catching up there is a long and difficult battle. Software, both large language models and robotics models, remains the bottleneck. Advancing the brain (AI models) is the fastest way to increase the usefulness of existing hardware and deployed systems. Second, the U.S. must 100x its investment in structured human judgment. Continued investment in compute and algorithmic efficiency is critical. But that investment is ultimately a bet on very high future inference demand. For that bet to pay off, models must unlock many new capabilities, and in practice the only way to unlock those capabilities is through expert human data. Historically, experts like doctors and lawyers were never incentivized to produce high-quality reasoning data in a machine-verifiable format. There was no reason for a doctor to generate precise, structured simulations of patient interactions, diagnostic reasoning, or treatment tradeoffs. There was no reason for a lawyer to document complex legal reasoning paths in a way that could be programmatically evaluated. AI systems now require exactly this kind of data. The incentive finally exists because this data directly improves systems that operate at massive scale, and experts can be paid well to produce it. Once expert judgment is encoded into models in a structured, verifiable way, it compounds. Those who delay do not just lose time. They lose the ability to catch up. Third, distillation from Chinese labs must be stopped. AI labs must do everything they can to prevent Chinese labs and models from distilling frontier models. Simply calling frontier APIs, or even interacting through UIs, lets Chinese model companies rapidly generate high-quality supervised fine-tuning datasets and close the gap at a fraction of the cost. This method does not put you at the frontier, but it does let you catch up quickly, which is what we saw with DeepSeek. The West significantly overreacted to DeepSeek’s headline capabilities, but underreacted to the underlying dynamic: frontier access itself becomes a training set at a fraction of the cost. Human data platforms also have a duty to help prevent this distillation. Lastly, the U.S.government should set the standard for AI Evaluation that leads to real production usage. AI agents are under-deployed relative to what the technology allows because they are probabilistic systems that require a fundamentally different QA approach than deterministic software. Generic QA is insufficient; safely shipping agents requires explicit evaluation frameworks that assess their full action space. Organizations must clearly define which functions an agent is allowed to perform, how quality is measured for each function, and which domain experts are qualified to judge outcomes. With these frameworks in place, agents can be rigorously tested using structured human data, deployed to production with confidence, and continuously improved over time. The U.S. government should be the first large enterprise to implement rigorous evaluation systems across every function. If the government leads on evaluation-driven deployment, adoption across the private sector accelerates naturally. This is how American workers become more powerful. Each worker operates digital or physical agents that expand their effective output. Recruiting, manufacturing, logistics, and other domains shift toward human judgment overseeing autonomous execution. Reshoring occurs because it becomes economically rational. Work becomes more meaningful. This is a race to determine who controls the intelligence layer of the global economy. And that must be us. 🇺🇸
0:48

The U.S. MUST win the AI race We’ve implemented a clear policy at micro1: we will only work with U.S. AI labs and its allies. We made this decision because the AI race is not just about better products. It is about who controls the intelligence layer of the global economy, and whether frontier capability is used to strengthen the free world or to empower adversarial states. AI will be the most important technology of our lifetime. In the fullness of time, it will automate most functions across the economy. Not just software tasks, but coordination, production, logistics, judgment, and execution. As those functions are automated, human time is freed up to invent new ones. Those new functions then become candidates for automation themselves. This loop compounds. As this trajectory continues, output per worker increases dramatically. Entire categories of work become cheaper and faster to perform. Manufacturing reshoring becomes economically viable not because of policy intervention, but because intelligent systems operated domestically outperform global labor arbitrage. Goods and services trend toward lower marginal cost, while distribution improves through better coordination of supply and demand. That is the upside. However, this is impossible without deep integration of intelligent systems. For AI to meaningfully automate real-world functions inside enterprises or governments, it needs full context of any given enterprise. That means read and write access to its core databases. There is no credible path to automating high-impact functions without granting frontier systems that level of access. If the United States does not win the AI race, enterprises eventually face a constrained choice. Either grant that access to Chinese models controlled by an adversarial government, or rely on sub-optimal intelligence to automate functions that still must be automated. Both outcomes are not acceptable. And ultimately, this becomes the greatest national security risk the United States has ever faced. AI models are trained by humans. The judgment embedded in pre-training data and especially in expert post-training data largely determines how a model behaves. While emergent behavior exists, a useful approximation is that a model reflects the weighted aggregate of the human judgment distilled into it. Assisting foreign actors—who will naturally prioritize expert tasks aligned with their own interests—to dominate data creation embeds those interests directly into the intelligence layer itself. Once encoded at scale, these interests propagate through every downstream applications that relies on that intelligence. Here’s how we win. First, leverage is in software. China is ahead in hardware for physically intelligent systems. Catching up there is a long and difficult battle. Software, both large language models and robotics models, remains the bottleneck. Advancing the brain (AI models) is the fastest way to increase the usefulness of existing hardware and deployed systems. Second, the U.S. must 100x its investment in structured human judgment. Continued investment in compute and algorithmic efficiency is critical. But that investment is ultimately a bet on very high future inference demand. For that bet to pay off, models must unlock many new capabilities, and in practice the only way to unlock those capabilities is through expert human data. Historically, experts like doctors and lawyers were never incentivized to produce high-quality reasoning data in a machine-verifiable format. There was no reason for a doctor to generate precise, structured simulations of patient interactions, diagnostic reasoning, or treatment tradeoffs. There was no reason for a lawyer to document complex legal reasoning paths in a way that could be programmatically evaluated. AI systems now require exactly this kind of data. The incentive finally exists because this data directly improves systems that operate at massive scale, and experts can be paid well to produce it. Once expert judgment is encoded into models in a structured, verifiable way, it compounds. Those who delay do not just lose time. They lose the ability to catch up. Third, distillation from Chinese labs must be stopped. AI labs must do everything they can to prevent Chinese labs and models from distilling frontier models. Simply calling frontier APIs, or even interacting through UIs, lets Chinese model companies rapidly generate high-quality supervised fine-tuning datasets and close the gap at a fraction of the cost. This method does not put you at the frontier, but it does let you catch up quickly, which is what we saw with DeepSeek. The West significantly overreacted to DeepSeek’s headline capabilities, but underreacted to the underlying dynamic: frontier access itself becomes a training set at a fraction of the cost. Human data platforms also have a duty to help prevent this distillation. Lastly, the U.S.government should set the standard for AI Evaluation that leads to real production usage. AI agents are under-deployed relative to what the technology allows because they are probabilistic systems that require a fundamentally different QA approach than deterministic software. Generic QA is insufficient; safely shipping agents requires explicit evaluation frameworks that assess their full action space. Organizations must clearly define which functions an agent is allowed to perform, how quality is measured for each function, and which domain experts are qualified to judge outcomes. With these frameworks in place, agents can be rigorously tested using structured human data, deployed to production with confidence, and continuously improved over time. The U.S. government should be the first large enterprise to implement rigorous evaluation systems across every function. If the government leads on evaluation-driven deployment, adoption across the private sector accelerates naturally. This is how American workers become more powerful. Each worker operates digital or physical agents that expand their effective output. Recruiting, manufacturing, logistics, and other domains shift toward human judgment overseeing autonomous execution. Reshoring occurs because it becomes economically rational. Work becomes more meaningful. This is a race to determine who controls the intelligence layer of the global economy. And that must be us. 🇺🇸

Ali Ansari

396,355 次观看 • 6 个月前

$AMD's heading to $5T MC LT| Lowest $/M tokens 🧵 The real reason why Institutions are FOMOing into AMD while other Semi stocks are underperforming ($NVDA $AVGO) Not Financial Advice! DYOR! Under Dr. Lisa Su’s leadership, AMD has transformed from a distant challenger into a formidable force in AI infrastructure, delivering the industry’s most compelling TCO story for high-volume inference. Her clear vision open ecosystems, aggressive annual roadmaps, rack-scale innovation, and relentless focus on tokens-per-dollar has positioned AMD’s Helios racks as the go-to solution for hyperscalers and AI natives struggling with exploding token costs, collapsing the cost down to $0.0003-$0.0005/M tokens. I will link various threads on this analysis to supply chain and wafer ratio if you are interested in understanding the full picture. In the last 3-4 months, explosive Agentic AI demand significantly increased Inference demand for Agentic AI models with 5-10 agents. If you are a listener of CNBC or Bloomberg, u should know enterprises and companies are complaining abt cost of token, and how it starts to spike up way too much to make sense. The fact that most data center today are run by $NVDA Chips, where the cost is way too high for Training or Inference. 1. Token cost Here are some quick comp, so u understand why $META OpenAI Anthropic $MSFT $AMZN Softbank $GOOGL and many more small to medium AI Natives are buying AMD CPUs and GPUs as much as they want, or pretty much AMD chips are sold out for the next 3-5 years. Inference (Cost per Million Tokens) ~$NVDA B200 / HGX: ~$0.02–$0.08 on optimized workloads (FP4/MXFP4, speculative decoding). Significant improvement over Hopper but still premium-priced. GB200 NVL72 rack-scale: $0.05–$0.25+ ~$AMD Helios Racks: $0.0003-$0.0005 per M tokens, dramatically lower than NVIDIA equivalents in owned infra. MI355X node-level: Up to 40% more tokens per dollar vs. competing solutions ( B200), driven by higher memory capacity (up to 288GB+ HBM), strong bandwidth, and lower acquisition costs. Training ~$NVDA Rubin Rack is estimated $0.7-$1.2/M Tokens ~$AMD Helios Rack is estimated $0.65-$1.0/M Tokens 2. Why Hyperscalers and AI Natives Are Choosing AMD Token consumption (especially Agentic) is outpacing even NVIDIA’s efficiency gains, making diversification mandatory for economic viability. Massive deals reflect this reality like $META, OpenAI, $MSFT, Softbank, $AMZN, Oracle, LumaAI, G42... Dr. Lisa Su’s Vision in Action: Since taking the helm, Su has driven AMD’s turnaround with disciplined execution, annual GPU cadence (MI300 → MI350 → MI400), full-stack software (ROCm 7), open ecosystems (UALink, OCP designs), and customer-centric rack-scale solutions like Helios. Her emphasis on “tokens per dollar” and TCO has turned AMD into the pragmatic choice for sustainable AI scaling. Power/Energy Efficiency: ~Helios Rack-level is estimated at 120kW-140kW with 50% more HBM4 where Inference and Training cost matter ~Rubin Rack-Level is estimated at 160kW-230kw AMD Helios shines in owned TCO, memory density, and energy flexibility at hyperscale. Cost to build 1GW data center 1GW Helios Rack full build is estimated $30-$35B 1GW Rubin Rack full build is estimated $45-$55B 3. Superior CPUs to pair with GPUs on massive scale 5-10-20GW Agentic AI. autonomous, multi-step workflows with orchestration, tool use, parallel agents, data movement, and enterprise integration has dramatically increased the importance of strong host CPUs alongside GPUs. This shifts the CPU-to-GPU ratio higher and makes balanced systems critical toward 1:1 to 5:1 as enterprises testing more than 5-10 agents. AMD EPYC Venice excels ~Leadership core density (up to 256 Zen 6 cores per socket) for running many agents in parallel, orchestration layers, and high-throughput control-plane tasks. ~Superior performance-per-core and power efficiency ( up to 2.1x higher perf/core and 2.26x better SPECpower vs. NVIDIA Grace in benchmarks). ~Tight integration in Helios: One Venice CPU + multiple MI450 GPUs per node, enabling efficient data feeding to GPUs ("zero-copy"), parallel execution, and full rack utilization for complex agentic loops. Hyperscalers (Meta, Microsoft, Amazon, Google, Softbank) and AI natives (OpenAI, Anthropic...) are adopting high-core EPYC at scale specifically for these agentic demands, as CPUs now handle a larger share of non-model work (orchestration, policy enforcement, tool calls). This complements AMD’s lower-cost GPUs for overall TCO wins. Conclusion: NVIDIA’s Vera Rubin cannot compete with a 2 years old EPYC Turin, but AMD under Dr. Lisa Su has engineered the lowest cost-per-million-tokens, highly competitive energy-efficient solutions, and superior CPU orchestration for agentic AI at scale with Helios. Dr. Su has championed this shift since at least 2023, foreseeing the rise of agentic workflows that demand far more orchestration, parallel agents, and balanced compute well before the industry fully embraced it. Her long-term vision of AI moving from simple prompts to always-on, multi-agent systems has driven AMD’s investments in high-core EPYC CPUs and integrated rack-scale solutions, perfectly positioning the company for today’s realities. Hyperscalers and AI natives effectively have no choice but to buy more AMD system for Agentic AI as leadership in economical, power-aware, high-volume internal + agentic use. However, due to supply constraints where Supply is far behind Demand, this makes multi-vendor reality along with in-house chips drive faster industry progress, lower overall costs, and better sustainability. Not Financial Advice! DYOR! Video source: Microsoft Build 2026
0:49

$AMD's heading to $5T MC LT| Lowest $/M tokens 🧵 The real reason why Institutions are FOMOing into AMD while other Semi stocks are underperforming ($NVDA $AVGO) Not Financial Advice! DYOR! Under Dr. Lisa Su’s leadership, AMD has transformed from a distant challenger into a formidable force in AI infrastructure, delivering the industry’s most compelling TCO story for high-volume inference. Her clear vision open ecosystems, aggressive annual roadmaps, rack-scale innovation, and relentless focus on tokens-per-dollar has positioned AMD’s Helios racks as the go-to solution for hyperscalers and AI natives struggling with exploding token costs, collapsing the cost down to $0.0003-$0.0005/M tokens. I will link various threads on this analysis to supply chain and wafer ratio if you are interested in understanding the full picture. In the last 3-4 months, explosive Agentic AI demand significantly increased Inference demand for Agentic AI models with 5-10 agents. If you are a listener of CNBC or Bloomberg, u should know enterprises and companies are complaining abt cost of token, and how it starts to spike up way too much to make sense. The fact that most data center today are run by $NVDA Chips, where the cost is way too high for Training or Inference. 1. Token cost Here are some quick comp, so u understand why $META OpenAI Anthropic $MSFT $AMZN Softbank $GOOGL and many more small to medium AI Natives are buying AMD CPUs and GPUs as much as they want, or pretty much AMD chips are sold out for the next 3-5 years. Inference (Cost per Million Tokens) ~$NVDA B200 / HGX: ~$0.02–$0.08 on optimized workloads (FP4/MXFP4, speculative decoding). Significant improvement over Hopper but still premium-priced. GB200 NVL72 rack-scale: $0.05–$0.25+ ~$AMD Helios Racks: $0.0003-$0.0005 per M tokens, dramatically lower than NVIDIA equivalents in owned infra. MI355X node-level: Up to 40% more tokens per dollar vs. competing solutions ( B200), driven by higher memory capacity (up to 288GB+ HBM), strong bandwidth, and lower acquisition costs. Training ~$NVDA Rubin Rack is estimated $0.7-$1.2/M Tokens ~$AMD Helios Rack is estimated $0.65-$1.0/M Tokens 2. Why Hyperscalers and AI Natives Are Choosing AMD Token consumption (especially Agentic) is outpacing even NVIDIA’s efficiency gains, making diversification mandatory for economic viability. Massive deals reflect this reality like $META, OpenAI, $MSFT, Softbank, $AMZN, Oracle, LumaAI, G42... Dr. Lisa Su’s Vision in Action: Since taking the helm, Su has driven AMD’s turnaround with disciplined execution, annual GPU cadence (MI300 → MI350 → MI400), full-stack software (ROCm 7), open ecosystems (UALink, OCP designs), and customer-centric rack-scale solutions like Helios. Her emphasis on “tokens per dollar” and TCO has turned AMD into the pragmatic choice for sustainable AI scaling. Power/Energy Efficiency: ~Helios Rack-level is estimated at 120kW-140kW with 50% more HBM4 where Inference and Training cost matter ~Rubin Rack-Level is estimated at 160kW-230kw AMD Helios shines in owned TCO, memory density, and energy flexibility at hyperscale. Cost to build 1GW data center 1GW Helios Rack full build is estimated $30-$35B 1GW Rubin Rack full build is estimated $45-$55B 3. Superior CPUs to pair with GPUs on massive scale 5-10-20GW Agentic AI. autonomous, multi-step workflows with orchestration, tool use, parallel agents, data movement, and enterprise integration has dramatically increased the importance of strong host CPUs alongside GPUs. This shifts the CPU-to-GPU ratio higher and makes balanced systems critical toward 1:1 to 5:1 as enterprises testing more than 5-10 agents. AMD EPYC Venice excels ~Leadership core density (up to 256 Zen 6 cores per socket) for running many agents in parallel, orchestration layers, and high-throughput control-plane tasks. ~Superior performance-per-core and power efficiency ( up to 2.1x higher perf/core and 2.26x better SPECpower vs. NVIDIA Grace in benchmarks). ~Tight integration in Helios: One Venice CPU + multiple MI450 GPUs per node, enabling efficient data feeding to GPUs ("zero-copy"), parallel execution, and full rack utilization for complex agentic loops. Hyperscalers (Meta, Microsoft, Amazon, Google, Softbank) and AI natives (OpenAI, Anthropic...) are adopting high-core EPYC at scale specifically for these agentic demands, as CPUs now handle a larger share of non-model work (orchestration, policy enforcement, tool calls). This complements AMD’s lower-cost GPUs for overall TCO wins. Conclusion: NVIDIA’s Vera Rubin cannot compete with a 2 years old EPYC Turin, but AMD under Dr. Lisa Su has engineered the lowest cost-per-million-tokens, highly competitive energy-efficient solutions, and superior CPU orchestration for agentic AI at scale with Helios. Dr. Su has championed this shift since at least 2023, foreseeing the rise of agentic workflows that demand far more orchestration, parallel agents, and balanced compute well before the industry fully embraced it. Her long-term vision of AI moving from simple prompts to always-on, multi-agent systems has driven AMD’s investments in high-core EPYC CPUs and integrated rack-scale solutions, perfectly positioning the company for today’s realities. Hyperscalers and AI natives effectively have no choice but to buy more AMD system for Agentic AI as leadership in economical, power-aware, high-volume internal + agentic use. However, due to supply constraints where Supply is far behind Demand, this makes multi-vendor reality along with in-house chips drive faster industry progress, lower overall costs, and better sustainability. Not Financial Advice! DYOR! Video source: Microsoft Build 2026

Mike

145,992 次观看 • 2 个月前

Multi-agent systems offer incredible potential and unprecedented risks. How do you solve for observability, failure mode analysis, and guardrailing in the era of agents? Today, we’re announcing our Agent Reliability platform to observe, evaluate, guardrail, and improve agents at scale. You can get started with the complete platform for trustworthy agentic AI today for free, and here’s how we’re solving some of the biggest challenges in agent reliability: - Observability redesigned for agents Trace views collapse under complex workflows, so we created the Graph View, Timeline View, and Conversation View to offer rich, intuitive visualizations of agent decisions, tool calls, and conversation flows. This multi-dimensional approach enables teams to pinpoint exactly where and why agents deviate or fail. - Automated Failure Mode Analysis with our new Insights Engine Our Insights Engine ingests your logs, metrics, and agent code to automatically surface nuanced failure modes and their root causes. But knowing the problem is not enough; you need to know how to fix it. Insights Engine delivers actionable fixes and can even apply them automatically. With adaptive learning, your insights become smarter and more relevant as your agents evolve. - Evaluating Agents Across Multiple Dimensions Agentic systems interact across complex pathways, and evaluating their performance requires new metrics that reflect this increasing complexity. To deliver comprehensive agentic measurements, we’ve added more out-of-the-box agent metrics like flow adherence, agent flow, agent efficiency, and more. For specialized domains and unique workflows, custom metrics powered by our new Luna-2 small language models can be rapidly designed and fine-tuned for your specific use case. - Real-Time Guardrails Powered by Luna-2 As AI agents become more autonomous and complex, failures like hallucinations or unsafe actions increase dramatically. Without real-time guardrails, these errors will hurt your user experience and brand reputation. Our Luna-2 family of small language models is purpose-built to provide low-latency, cost-effective guardrails that actively stop agent errors before they happen. With support for out-of-the-box and custom metrics, Luna-2 enables enterprises to enforce safety, compliance, and reliability at scale. Enterprises running hundreds of agents and processing hundreds of millions of queries daily already rely on Galileo’s Agent Reliability platform to protect their users, safeguard brand trust, and accelerate innovation. Agent Reliability is available starting today. Try it for free and experience the new standard in AI reliability. Learn more below 👇
2:43

Multi-agent systems offer incredible potential and unprecedented risks. How do you solve for observability, failure mode analysis, and guardrailing in the era of agents? Today, we’re announcing our Agent Reliability platform to observe, evaluate, guardrail, and improve agents at scale. You can get started with the complete platform for trustworthy agentic AI today for free, and here’s how we’re solving some of the biggest challenges in agent reliability: - Observability redesigned for agents Trace views collapse under complex workflows, so we created the Graph View, Timeline View, and Conversation View to offer rich, intuitive visualizations of agent decisions, tool calls, and conversation flows. This multi-dimensional approach enables teams to pinpoint exactly where and why agents deviate or fail. - Automated Failure Mode Analysis with our new Insights Engine Our Insights Engine ingests your logs, metrics, and agent code to automatically surface nuanced failure modes and their root causes. But knowing the problem is not enough; you need to know how to fix it. Insights Engine delivers actionable fixes and can even apply them automatically. With adaptive learning, your insights become smarter and more relevant as your agents evolve. - Evaluating Agents Across Multiple Dimensions Agentic systems interact across complex pathways, and evaluating their performance requires new metrics that reflect this increasing complexity. To deliver comprehensive agentic measurements, we’ve added more out-of-the-box agent metrics like flow adherence, agent flow, agent efficiency, and more. For specialized domains and unique workflows, custom metrics powered by our new Luna-2 small language models can be rapidly designed and fine-tuned for your specific use case. - Real-Time Guardrails Powered by Luna-2 As AI agents become more autonomous and complex, failures like hallucinations or unsafe actions increase dramatically. Without real-time guardrails, these errors will hurt your user experience and brand reputation. Our Luna-2 family of small language models is purpose-built to provide low-latency, cost-effective guardrails that actively stop agent errors before they happen. With support for out-of-the-box and custom metrics, Luna-2 enables enterprises to enforce safety, compliance, and reliability at scale. Enterprises running hundreds of agents and processing hundreds of millions of queries daily already rely on Galileo’s Agent Reliability platform to protect their users, safeguard brand trust, and accelerate innovation. Agent Reliability is available starting today. Try it for free and experience the new standard in AI reliability. Learn more below 👇

Galileo

1,276,298 次观看 • 1 年前

$AMD Massive Rotation from $NVDA $INTC🧵 Not Financial Advice! DYOR! 5-10 minutes before the bell today, last trading day of May 2026, massive rotation out of $INTC and $NVDA into $AMD. I wrote this thread this morning on what $TSM said on Energy Efficiency is now TOP Priotity and why AMD is the biggest winner. Of course I did not have influence on this rebalancing, I was just pointing out why Dr. Su saw this coming years ago. (Check the picture to understand more). I been talking about Agentic AI for like 3-4 years now. OpenClaw broke the CPU:GPU Ratio 1:4 narrative to 1:1 to 5:1 in late Jan and Feb 2026. I will link various threads where you can understand the full picture from supply chain, to TSMC expansion, and different Wafer Ratio for EPYC Venice and MI455X. Energy efficiency is a structural, long-term driver behind institutional rotation from $NVDA and $INTC into $AMD (with spillover strength in $AVGO for complementary networking/custom silicon). This isn't just short-term rebalancing, it's a massive bet on the shift from AI training (performance-at-any-cost) to inference, deployment, and embodied/agentic systems (where total cost of ownership, power draw, and scalability dominate). Precisely What I been writing about $AMD for years now, probably at least more than 5,000 threads.This is the FOMO from Institutions to own $AMD. Do know that AMD is the least owned Semi Stock among vs Peers. AI infrastructure is moving beyond massive training clusters to widespread inference for Agentic AI (running models 24/7) and embodied AI (robots, autonomous agents, edge devices). These workloads prioritize: ~Tokens-per-watt and performance-per-watt ~Lower total power consumption for data centers facing grid constraints ~Better economics at scale (cost-per-token, TCO) ~Thermal and power efficiency for on-device/robotics use Hyperscalers are now thinking more about Margin, Profitability, and $/M Tokens At $516/share. AMD Fwd PEG Ratio is still 35/100+= 0.35 AKA very cheap IMO for the growth and potential. A. Why institutions rotated out of $NVDA? Because Agentic AI is going to dominated by CPUs for years to come, moving violently to 5-10-20:1 CPU:GPU Ratio as enterprises are demanding more than 10-20 agents to run tasks. Now, that does not mean training is going away, Inference is just going to grow much faster. B. Why instiutitons rotated out of $INTC? Because AMD x86 unit share is only at 30-31% but Revenue share is already at 46.2% according to Mercury Research. And Dr. Su wants 50-60% market share, and that would mean 60-70%+ Revenue share where the CPUs TAM Is now already at $200B in 2026 and projected to be $500B by 2030. C. Why $AMD? Because AMD secured meaningful 2nm Capacity, Advanced Packaging and Memory through 2027-2028. And TSMC is expanding 2 primary 2nm Fabs toward 60-65k WPM each, and speeding up 5 2nm Fabs in Taiwan. With total up to 12 2nm Fabs through 2027/2028. 2nm Capacity is expected to be 140k+ WPM toward end of 2026, and 220-240k WPM by end of 2027. Apple has secured 35-45k WPM. And AMD does not have to worry about allocation competition until late 2027 from $AVGO for $META and $GOOGL(This may change) D. Agentic AI will evolve to 24/7 Autonomous Agent, and that will become the foundational layer for Robotic or Physical AI. Agentic AI (autonomous systems that plan, reason, use tools, self-correct, pursue long-horizon goals, and adapt) provides the high-level cognitive architecture. It turns raw perception and low-level control into useful, general-purpose behavior in the physical world. Physical AI (or Embodied AI) refers to AI that senses, understands, and acts directly in the real world through robots, actuators, and sensors. Agentic capabilities are what make this scalable and useful beyond narrow, scripted tasks. Reactive/programmed machines → To proactive, goal-oriented autonomous agents. How does this work? Autonomous Agent layer is the brain ~Vision-Language-Action models or robotics foundation models. ~Agentic loops: Planning, chain-of-thought reasoning, reflection, tool use (simulators, APIs), multi-step task decomposition. ~Persistent 24/7 operation with Memory, world modeling, continuous learning. Institutions may not like $AMD from 2022-2025, but they cannot stop this evolution and it is inevitable. Part of my main thesis for AMD to get to $5 Trillion Market Cap Long Term. Conclusion: Institutions are rotating capital toward AMD not merely for tactical rebalancing, but because Dr. Lisa Su and her team anticipated this exact inflection years in advance and have been methodically engineering AMD’s platform to dominate it. Dr. Su has long championed the convergence of Agentic AI as the high-level cognitive foundation for Physical AI and robotics. As far back as her 2023/2024 CES keynote and earlier strategic commentary, she described Physical AI (including humanoid robotics and edge autonomy) as “the next big thing”; a natural extension of agentic workflows moving from digital reasoning to real-world action. She emphasized that enabling persistent, 24/7 autonomous agents requires a full-stack approach: high-performance CPUs for orchestration and motion control, dedicated accelerators for real-time vision and multimodal inference, and open software ecosystems for rapid development. This vision aligns precisely with the structural drivers we’ve discussed. As AI shifts from training to massive-scale inference and embodiment, energy efficiency, total cost of ownership, and heterogeneous compute become first-order advantages. AMD’s Instinct MI350/MI355 series, Ryzen AI Embedded processors, and EPYC platforms deliver superior performance-per-watt and balanced CPU + GPU + NPU integration ideal for power-constrained robots that must run sophisticated agentic reasoning loops without excessive thermal or battery drain. Dr. Su has repeatedly highlighted the rising importance of CPUs in agentic systems (moving toward 1:1 or even CPU-heavy ratios with GPUs), positioning AMD’s strengths in orchestration, memory handling, and efficiency as critical for the next phase of growth. AMD is engineered for the deployment realities of embodied agents: scalable, efficient, and deployable at the edge and in physical systems. The institutional flows out of NVDA and INTC into AMD reflect recognition of this prepared leadership. Dr. Su didn’t just see the future of Agentic AI powering robotics, she has spent years building the silicon, software, and partnerships to make it practical and economically viable. This rotation signals confidence that the companies best positioned for the physical, always-on intelligence layer will capture the highest-volume opportunities in the coming decade. Not Financial Advice! DYOR!
2:25

$AMD Massive Rotation from $NVDA $INTC🧵 Not Financial Advice! DYOR! 5-10 minutes before the bell today, last trading day of May 2026, massive rotation out of $INTC and $NVDA into $AMD. I wrote this thread this morning on what $TSM said on Energy Efficiency is now TOP Priotity and why AMD is the biggest winner. Of course I did not have influence on this rebalancing, I was just pointing out why Dr. Su saw this coming years ago. (Check the picture to understand more). I been talking about Agentic AI for like 3-4 years now. OpenClaw broke the CPU:GPU Ratio 1:4 narrative to 1:1 to 5:1 in late Jan and Feb 2026. I will link various threads where you can understand the full picture from supply chain, to TSMC expansion, and different Wafer Ratio for EPYC Venice and MI455X. Energy efficiency is a structural, long-term driver behind institutional rotation from $NVDA and $INTC into $AMD (with spillover strength in $AVGO for complementary networking/custom silicon). This isn't just short-term rebalancing, it's a massive bet on the shift from AI training (performance-at-any-cost) to inference, deployment, and embodied/agentic systems (where total cost of ownership, power draw, and scalability dominate). Precisely What I been writing about $AMD for years now, probably at least more than 5,000 threads.This is the FOMO from Institutions to own $AMD. Do know that AMD is the least owned Semi Stock among vs Peers. AI infrastructure is moving beyond massive training clusters to widespread inference for Agentic AI (running models 24/7) and embodied AI (robots, autonomous agents, edge devices). These workloads prioritize: ~Tokens-per-watt and performance-per-watt ~Lower total power consumption for data centers facing grid constraints ~Better economics at scale (cost-per-token, TCO) ~Thermal and power efficiency for on-device/robotics use Hyperscalers are now thinking more about Margin, Profitability, and $/M Tokens At $516/share. AMD Fwd PEG Ratio is still 35/100+= 0.35 AKA very cheap IMO for the growth and potential. A. Why institutions rotated out of $NVDA? Because Agentic AI is going to dominated by CPUs for years to come, moving violently to 5-10-20:1 CPU:GPU Ratio as enterprises are demanding more than 10-20 agents to run tasks. Now, that does not mean training is going away, Inference is just going to grow much faster. B. Why instiutitons rotated out of $INTC? Because AMD x86 unit share is only at 30-31% but Revenue share is already at 46.2% according to Mercury Research. And Dr. Su wants 50-60% market share, and that would mean 60-70%+ Revenue share where the CPUs TAM Is now already at $200B in 2026 and projected to be $500B by 2030. C. Why $AMD? Because AMD secured meaningful 2nm Capacity, Advanced Packaging and Memory through 2027-2028. And TSMC is expanding 2 primary 2nm Fabs toward 60-65k WPM each, and speeding up 5 2nm Fabs in Taiwan. With total up to 12 2nm Fabs through 2027/2028. 2nm Capacity is expected to be 140k+ WPM toward end of 2026, and 220-240k WPM by end of 2027. Apple has secured 35-45k WPM. And AMD does not have to worry about allocation competition until late 2027 from $AVGO for $META and $GOOGL(This may change) D. Agentic AI will evolve to 24/7 Autonomous Agent, and that will become the foundational layer for Robotic or Physical AI. Agentic AI (autonomous systems that plan, reason, use tools, self-correct, pursue long-horizon goals, and adapt) provides the high-level cognitive architecture. It turns raw perception and low-level control into useful, general-purpose behavior in the physical world. Physical AI (or Embodied AI) refers to AI that senses, understands, and acts directly in the real world through robots, actuators, and sensors. Agentic capabilities are what make this scalable and useful beyond narrow, scripted tasks. Reactive/programmed machines → To proactive, goal-oriented autonomous agents. How does this work? Autonomous Agent layer is the brain ~Vision-Language-Action models or robotics foundation models. ~Agentic loops: Planning, chain-of-thought reasoning, reflection, tool use (simulators, APIs), multi-step task decomposition. ~Persistent 24/7 operation with Memory, world modeling, continuous learning. Institutions may not like $AMD from 2022-2025, but they cannot stop this evolution and it is inevitable. Part of my main thesis for AMD to get to $5 Trillion Market Cap Long Term. Conclusion: Institutions are rotating capital toward AMD not merely for tactical rebalancing, but because Dr. Lisa Su and her team anticipated this exact inflection years in advance and have been methodically engineering AMD’s platform to dominate it. Dr. Su has long championed the convergence of Agentic AI as the high-level cognitive foundation for Physical AI and robotics. As far back as her 2023/2024 CES keynote and earlier strategic commentary, she described Physical AI (including humanoid robotics and edge autonomy) as “the next big thing”; a natural extension of agentic workflows moving from digital reasoning to real-world action. She emphasized that enabling persistent, 24/7 autonomous agents requires a full-stack approach: high-performance CPUs for orchestration and motion control, dedicated accelerators for real-time vision and multimodal inference, and open software ecosystems for rapid development. This vision aligns precisely with the structural drivers we’ve discussed. As AI shifts from training to massive-scale inference and embodiment, energy efficiency, total cost of ownership, and heterogeneous compute become first-order advantages. AMD’s Instinct MI350/MI355 series, Ryzen AI Embedded processors, and EPYC platforms deliver superior performance-per-watt and balanced CPU + GPU + NPU integration ideal for power-constrained robots that must run sophisticated agentic reasoning loops without excessive thermal or battery drain. Dr. Su has repeatedly highlighted the rising importance of CPUs in agentic systems (moving toward 1:1 or even CPU-heavy ratios with GPUs), positioning AMD’s strengths in orchestration, memory handling, and efficiency as critical for the next phase of growth. AMD is engineered for the deployment realities of embodied agents: scalable, efficient, and deployable at the edge and in physical systems. The institutional flows out of NVDA and INTC into AMD reflect recognition of this prepared leadership. Dr. Su didn’t just see the future of Agentic AI powering robotics, she has spent years building the silicon, software, and partnerships to make it practical and economically viable. This rotation signals confidence that the companies best positioned for the physical, always-on intelligence layer will capture the highest-volume opportunities in the coming decade. Not Financial Advice! DYOR!

Mike

104,109 次观看 • 2 个月前

$AMD| The FOMO to buy AMD Chips is NOW 🧵 Not Financial Advice! DYOR! Research Purpose Only! The Inference Queen is the biggest winner in Agentic AI where all other CPUs are struggling to compete with a 2yr old EPYC Turin and EPYC Venice is in mass production phase. AMD stresses deployability today on standard x86 platforms (no proprietary architectures required), full software compatibility, and open standards. This positions Venice + Helios as a practical, high-density alternative to competing solutions while underscoring that agentic AI shifts the balance toward CPU-rich racks alongside GPUs, and most importantly, lowering the cost of token to accelerate adoption and innovation. Context: The Wall Street Journal yesterday came out with an article that OpenAI is condiering drasstically lowering the token prices to win more customers from Anthropic. The narrative "they" are trying to exacerbate the current AI selloff won't last long. This is a fundamental misunderstanding of what is going on, or what I already discussed for months and years. Followers and Subscribers already knew this for years, that this day would come, where token cost will bcome the central discussion among enterprises as there is no such thing as unlimited budget or Tokenmaxxing when they use $NVDA chips or In-house Hyperscalers chips. I will link various threads if you are interested in understanding the full picture from supply chain to recent TSMC Rapid 2nm expansion up to 12 Fabs total by 2027/2028. Hyperscalers and AI natives effectively have no choice but to buy more AMD system for Agentic AI as leadership in economical, power-aware, high-volume internal + agentic use. However, due to supply constraints where Supply is far behind Demand, this makes multi-vendor reality along with in-house chips drive faster industry progress, lower overall costs, and better sustainability. NVIDIA’s Vera Rubin cannot compete with a 2 years old EPYC Turin, but AMD under Dr. Lisa Su has engineered the lowest cost-per-million-tokens, highly competitive energy-efficient solutions, and superior CPU orchestration for agentic AI at scale with Helios. Dr. Su has championed this shift since at least 2023, foreseeing the rise of agentic workflows that demand far more orchestration, parallel agents, and balanced compute well before the industry fully embraced it. Her long-term vision of AI moving from simple prompts to always on, multi-agent systems has driven AMD’s investments in high-core EPYC CPUs and integrated rack-scale solutions, perfectly positioning the company for today’s realities. The OpenAI-AMD 1GW Helios deployment (starting H2 2026) represents a pivotal vertical integration move that directly supercharges the inference economics. This isn't incremental; it's a structural shift toward ownership of massive, optimized rack-scale capacity, enabling the lowest token costs and triggering the enterprise adoption flywheel. We need to be honest, $AMD is the only company that made a big bet on Inference since the day Chatgpt became sensational where $NVDA and others were betting big on Training. At the end of the day, Token bill from Anthropic has to obey economics. Meaning the bills rise, companies have to get more out of it to justify the cost. It cannot be an unlimited inference budget, and it has to show up on efficiency, profitability and operating leverage. 1. Tokenomics After you understand this, you will understand why Citi cited Anthropic is likely to sign a deal with $AMD along with Hyperscalers, AI Labs, Sovereign AI like Softbank 5GW in France and many other countries. However, OpenAI and $META are now wanting faster deployment, and they are AMD shareholders now, they have prioritized allocation. Anthropic and Hyperscalers just cannot compete when Helios Rack lower token cost to$0.0003–$0.0005 per million tokens at GW scale. Cost to build 1GW data center 1GW Helios Rack full build is estimated $30-$35B 1GW Rubin Rack full build is estimated $45-$55B Inference (Cost per Million Tokens) ~$NVDA B200 / HGX: ~$0.02–$0.08 on optimized workloads (FP4/MXFP4, speculative decoding). Significant improvement over Hopper but still premium-priced. GB200 NVL72 rack-scale: $0.05–$0.25+ ~$AMD Helios Racks: $0.0003-$0.0005 per M tokens, dramatically lower than NVIDIA equivalents in owned infra. MI355X node-level: Up to 40% more tokens per dollar vs. competing solutions ( B200), driven by higher memory capacity (up to 288GB+ HBM), strong bandwidth, and lower acquisition costs. Training ~$NVDA Rubin Rack is estimated $0.7-$1.2/M Tokens ~$AMD Helios Rack is estimated $0.65-$1.0/M Tokens Now, OpenAI, META and Hyperscalers can lower Inference cost even further with $AMD EPYC Venice "dense rack" or Agentic AI Rack. AMD published a detailed technical blog emphasizing that the future of agentic AI autonomous, multi-step AI systems requiring heavy orchestration, databases, caching, APIs, and control planes demands massive CPU-dense rack-scale infrastructure, not just GPUs. The catalyst prominently positions their upcoming 6th Gen EPYC "Venice" processors as the key enabler for next-generation dense racks, delivering leadership throughput under real-world power, cooling, and density constraints. ~EPYC Venice (Zen 6 architecture, up to 256 cores / 512 threads per socket) is projected to deliver exceptional rack-level performance. In AMD’s modeled 100 kW rack comparisons, Venice-powered systems are expected to achieve ~3.30x the throughput of NVIDIA’s Vera (88-core Olympus) baseline across a broad mix of agentic-supporting workloads. ~This builds on current-generation 5th Gen EPYC "Turin" (up to 192 cores), which already delivers ~2.37x rack throughput vs. Vera and ~1.6x vs. Intel’s Xeon 6980P (128 cores). ~ Liquid-cooled Turin deployments already support >27,000 CPU cores per rack today. Venice is architected to push this beyond 36,000 cores in the same rack class, dramatically increasing concurrent agent capacity and overall infrastructure efficiency. 2. Ownership vs renting compute from Hyperscalers matter to OpenAI and only owning $AMD chips can meaningfully lower token cost for enterprises. ~Eliminates cloud overhead: No provider margins, utilization buffers, or egress fees. Direct control over power contracts, cooling, scheduling, and orchestration at dedicated facilities. ~Helios optimizations at GW scale: Rack-level density (1.4+ exaFLOPS FP8 per rack), high HBM4 bandwidth, EPYC orchestration for agentic workloads, and superior TCO/TDP. AMD's long-standing focus on tokens per dollar/watt shines here 20-40%+ efficiency edges in inference-heavy scenarios. ~At 1GW+ optimized deployment, inference hits $0.0003–$0.0005 per million tokens (community/analyst models tied to Helios metrics). This is dramatically lower than typical rented/cloud equivalents, especially for high-volume output tokens in agentic flows. High token bills today, enterprises running heavy agentic/coding/analysis workloads can face $50-100M+/month at current API rates (flagship models $5-30+/M output, scaled to massive volumes). Post-Helios compression, same volume will drop to $10-15M/month (or better) via lower underlying costs passed through as pricing flexibility, volume tiers, caching, or batch discounts. ROI thresholds collapse. More companies greenlight pilots → production → massive scaling. Agentic AI (autonomous workflows) multiplies token demand exponentially, but affordability removes the friction. OpenAI gains flexibility, Unlike more cloud-dependent rivals (Anthropic), they can lower effective pricing, offer aggressive enterprise bundles, or absorb volume without margin destruction directly tackling "high token bill" complaints while maintaining profitability as usage explodes. 3. Agentic AI Models shifted CPU:GPU Ratio to 1:1 toward 3-5:1 with Explosively Token-Hungry Workloads Agentic AI (autonomous, multi-step agents with planning, tool use, iteration, and self-correction) is fundamentally more compute and token intensive than conversational or single-turn generative AI. Agentic AI. autonomous, multi-step workflows with orchestration, tool use, parallel agents, data movement, and enterprise integration has dramatically increased the importance of strong host CPUs alongside GPUs. This shifts the CPU-to-GPU ratio higher and makes balanced systems critical toward 1:1 to 5:1 as enterprises testing more than 5-10 agents. AMD EPYC Venice excels ~Leadership core density (up to 256 Zen 6 cores per socket) for running many agents in parallel, orchestration layers, and high-throughput control-plane tasks. ~Superior performance-per-core and power efficiency ( up to 2.1x higher perf/core and 2.26x better SPECpower vs. NVIDIA Grace in benchmarks). ~Tight integration in Helios: One Venice CPU + multiple MI450 GPUs per node, enabling efficient data feeding to GPUs ("zero-copy"), parallel execution, and full rack utilization for complex agentic loops. Hyperscalers (Meta, Microsoft, Amazon, Google, Softbank) and AI natives (OpenAI, Anthropic...) are adopting high-core EPYC at scale specifically for these agentic demands, as CPUs now handle a larger share of non-model work (orchestration, policy enforcement, tool calls). This complements AMD’s lower-cost GPUs for overall TCO wins. ~Agents often generate 10–100x+ more tokens per task due to iterative reasoning chains, multiple tool calls, verification loops, and long-context orchestration. ~Goldman Sachs forecasts token consumption multiplying 24x by 2030 (to 120 quadrillion tokens/month) largely driven by agentic adoption in consumer and enterprise. ~Enterprise data shows agent-pattern workloads growing at 680% annualized rates, projected to surpass conversational AI in token volume by Q3 2026. ~Daily enterprise agent token consumption is already in the billions, with complex workflows (coding, workflows, analysis) amplifying this dramatically. 4. Competitive Edge: Winning Customers from Anthropic Anthropic’s Claude models (especially Opus/Sonnet) excel in complex reasoning and agentic coding, commanding premium positioning. However, their higher underlying costs (heavier reliance on third-party cloud with margins) limit pricing flexibility compared to OpenAI’s owned Helios capacity. Anthropic is on track to generate $10.9 billion in Q2 revenue. The company expects to achieve its first-ever quarterly adjusted operating profit of $559 million. However, sustaining full-year profitability remains challenging due to immense computing and model training costs The truth is, Anthropic has no choice but to buy as much $AMD chips as possible if they want to compete with OpenAI or get investors attention. This 5% adjusted operating profit to revenue ratio is just pathetic. Current pricing dynamics (2026): OpenAI already undercuts on many tiers ( flagship output tokens significantly cheaper than equivalent Claude Opus). Nano/mini models offer 5–10x advantages for volume work. Anthropic holds edges in long-context flat pricing and certain reasoning quality. OpenAI after Helios Rack Ownership, At $0.0003–$0.0005/M effective costs, OpenAI gains massive headroom to: ~Aggressively discount high-volume agentic tiers or bundles. ~Offer “unlimited” enterprise plans or usage-based models that Anthropic struggles to match without margin erosion. ~Target cost-sensitive, high-throughput agent deployments (dev tools, automation platforms) where token bills explode. Enterprises facing $ millions in monthly agentic bills will migrate to the provider delivering better economics at scale. OpenAI’s combination of strong models (o-series reasoning) + lowest TCO positions it to erode Anthropic’s enterprise share, especially as agentic becomes the dominant token consumer. Cheaper tokens expand the total addressable market dramatically. This feeds the data/model improvement loop, justifying further capex. AMD benefits from proven scale pulling in more customers (Meta, Oracle, Microsfot, Amazon, Softbank, TensorWave, LumaAI ... already aligned on Helios). Conclusion: Dr. Lisa Su has been laser focused on inference economics since at least 2022–2023, repeatedly emphasizing that the real battleground for AI scalability would be TCO, power efficiency (TDP), and ultimately tokens per dollar and per watt not just raw training FLOPS. While many viewed inference as a secondary, commoditized workload, Dr. Su architected AMD’s roadmap around rack-scale systems optimized for high-volume, sustained inference that would dominate as models matured and usage exploded. Helios represents the culmination of that multi-year bet: a fully integrated, open platform designed precisely for the economics of massive token throughput. This deep, strategic partnership with OpenAI starting with the 1GW Helios deployment in H2 2026 and scaling to 6GW, is the embodiment of that shared vision. Both companies foresaw a future where agentic AI models evolve to become extraordinarily token-hungry: autonomous agents executing complex, iterative workflows with planning, tool use, verification loops, and long-context reasoning. These workloads can consume 100x+ more tokens per task than traditional chat or single-turn generation, driving exponential demand as capabilities improve and enterprises deploy them at scale. By owning and optimizing this massive Helios capacity at GW scale, OpenAI achieves inference costs as low as $0.0003–$0.0005 per million tokens. This structural cost advantage allows OpenAI to absorb the coming token explosion profitably, dramatically lower effective pricing for enterprises, and win high-volume agentic workloads from higher-cost competitors like Anthropic. What was once a prohibitive monthly token bill becomes an affordable accelerator for productivity and innovation. The OpenAI-AMD alliance validates Dr. Su’s prescient strategy and turns the Agentic flywheel into reality: Collapsing inference costs → explosive token consumption → richer data and better models → accelerate greater demand. This partnership doesn’t just address today’s economics, it positions both leaders at the center of the infrastructure buildout that will power AI’s next decade. By delivering the lowest inference economics at scale, OpenAI not only solves enterprise bill pain but gains a decisive weapon to win share from higher-cost rivals like Anthropic. And that is why OpenAI and $META will deploy EPYC Dense Rack Not Financial Advice! DYOR! Research Purpose Only!
4:46

$AMD| The FOMO to buy AMD Chips is NOW 🧵 Not Financial Advice! DYOR! Research Purpose Only! The Inference Queen is the biggest winner in Agentic AI where all other CPUs are struggling to compete with a 2yr old EPYC Turin and EPYC Venice is in mass production phase. AMD stresses deployability today on standard x86 platforms (no proprietary architectures required), full software compatibility, and open standards. This positions Venice + Helios as a practical, high-density alternative to competing solutions while underscoring that agentic AI shifts the balance toward CPU-rich racks alongside GPUs, and most importantly, lowering the cost of token to accelerate adoption and innovation. Context: The Wall Street Journal yesterday came out with an article that OpenAI is condiering drasstically lowering the token prices to win more customers from Anthropic. The narrative "they" are trying to exacerbate the current AI selloff won't last long. This is a fundamental misunderstanding of what is going on, or what I already discussed for months and years. Followers and Subscribers already knew this for years, that this day would come, where token cost will bcome the central discussion among enterprises as there is no such thing as unlimited budget or Tokenmaxxing when they use $NVDA chips or In-house Hyperscalers chips. I will link various threads if you are interested in understanding the full picture from supply chain to recent TSMC Rapid 2nm expansion up to 12 Fabs total by 2027/2028. Hyperscalers and AI natives effectively have no choice but to buy more AMD system for Agentic AI as leadership in economical, power-aware, high-volume internal + agentic use. However, due to supply constraints where Supply is far behind Demand, this makes multi-vendor reality along with in-house chips drive faster industry progress, lower overall costs, and better sustainability. NVIDIA’s Vera Rubin cannot compete with a 2 years old EPYC Turin, but AMD under Dr. Lisa Su has engineered the lowest cost-per-million-tokens, highly competitive energy-efficient solutions, and superior CPU orchestration for agentic AI at scale with Helios. Dr. Su has championed this shift since at least 2023, foreseeing the rise of agentic workflows that demand far more orchestration, parallel agents, and balanced compute well before the industry fully embraced it. Her long-term vision of AI moving from simple prompts to always on, multi-agent systems has driven AMD’s investments in high-core EPYC CPUs and integrated rack-scale solutions, perfectly positioning the company for today’s realities. The OpenAI-AMD 1GW Helios deployment (starting H2 2026) represents a pivotal vertical integration move that directly supercharges the inference economics. This isn't incremental; it's a structural shift toward ownership of massive, optimized rack-scale capacity, enabling the lowest token costs and triggering the enterprise adoption flywheel. We need to be honest, $AMD is the only company that made a big bet on Inference since the day Chatgpt became sensational where $NVDA and others were betting big on Training. At the end of the day, Token bill from Anthropic has to obey economics. Meaning the bills rise, companies have to get more out of it to justify the cost. It cannot be an unlimited inference budget, and it has to show up on efficiency, profitability and operating leverage. 1. Tokenomics After you understand this, you will understand why Citi cited Anthropic is likely to sign a deal with $AMD along with Hyperscalers, AI Labs, Sovereign AI like Softbank 5GW in France and many other countries. However, OpenAI and $META are now wanting faster deployment, and they are AMD shareholders now, they have prioritized allocation. Anthropic and Hyperscalers just cannot compete when Helios Rack lower token cost to$0.0003–$0.0005 per million tokens at GW scale. Cost to build 1GW data center 1GW Helios Rack full build is estimated $30-$35B 1GW Rubin Rack full build is estimated $45-$55B Inference (Cost per Million Tokens) ~$NVDA B200 / HGX: ~$0.02–$0.08 on optimized workloads (FP4/MXFP4, speculative decoding). Significant improvement over Hopper but still premium-priced. GB200 NVL72 rack-scale: $0.05–$0.25+ ~$AMD Helios Racks: $0.0003-$0.0005 per M tokens, dramatically lower than NVIDIA equivalents in owned infra. MI355X node-level: Up to 40% more tokens per dollar vs. competing solutions ( B200), driven by higher memory capacity (up to 288GB+ HBM), strong bandwidth, and lower acquisition costs. Training ~$NVDA Rubin Rack is estimated $0.7-$1.2/M Tokens ~$AMD Helios Rack is estimated $0.65-$1.0/M Tokens Now, OpenAI, META and Hyperscalers can lower Inference cost even further with $AMD EPYC Venice "dense rack" or Agentic AI Rack. AMD published a detailed technical blog emphasizing that the future of agentic AI autonomous, multi-step AI systems requiring heavy orchestration, databases, caching, APIs, and control planes demands massive CPU-dense rack-scale infrastructure, not just GPUs. The catalyst prominently positions their upcoming 6th Gen EPYC "Venice" processors as the key enabler for next-generation dense racks, delivering leadership throughput under real-world power, cooling, and density constraints. ~EPYC Venice (Zen 6 architecture, up to 256 cores / 512 threads per socket) is projected to deliver exceptional rack-level performance. In AMD’s modeled 100 kW rack comparisons, Venice-powered systems are expected to achieve ~3.30x the throughput of NVIDIA’s Vera (88-core Olympus) baseline across a broad mix of agentic-supporting workloads. ~This builds on current-generation 5th Gen EPYC "Turin" (up to 192 cores), which already delivers ~2.37x rack throughput vs. Vera and ~1.6x vs. Intel’s Xeon 6980P (128 cores). ~ Liquid-cooled Turin deployments already support >27,000 CPU cores per rack today. Venice is architected to push this beyond 36,000 cores in the same rack class, dramatically increasing concurrent agent capacity and overall infrastructure efficiency. 2. Ownership vs renting compute from Hyperscalers matter to OpenAI and only owning $AMD chips can meaningfully lower token cost for enterprises. ~Eliminates cloud overhead: No provider margins, utilization buffers, or egress fees. Direct control over power contracts, cooling, scheduling, and orchestration at dedicated facilities. ~Helios optimizations at GW scale: Rack-level density (1.4+ exaFLOPS FP8 per rack), high HBM4 bandwidth, EPYC orchestration for agentic workloads, and superior TCO/TDP. AMD's long-standing focus on tokens per dollar/watt shines here 20-40%+ efficiency edges in inference-heavy scenarios. ~At 1GW+ optimized deployment, inference hits $0.0003–$0.0005 per million tokens (community/analyst models tied to Helios metrics). This is dramatically lower than typical rented/cloud equivalents, especially for high-volume output tokens in agentic flows. High token bills today, enterprises running heavy agentic/coding/analysis workloads can face $50-100M+/month at current API rates (flagship models $5-30+/M output, scaled to massive volumes). Post-Helios compression, same volume will drop to $10-15M/month (or better) via lower underlying costs passed through as pricing flexibility, volume tiers, caching, or batch discounts. ROI thresholds collapse. More companies greenlight pilots → production → massive scaling. Agentic AI (autonomous workflows) multiplies token demand exponentially, but affordability removes the friction. OpenAI gains flexibility, Unlike more cloud-dependent rivals (Anthropic), they can lower effective pricing, offer aggressive enterprise bundles, or absorb volume without margin destruction directly tackling "high token bill" complaints while maintaining profitability as usage explodes. 3. Agentic AI Models shifted CPU:GPU Ratio to 1:1 toward 3-5:1 with Explosively Token-Hungry Workloads Agentic AI (autonomous, multi-step agents with planning, tool use, iteration, and self-correction) is fundamentally more compute and token intensive than conversational or single-turn generative AI. Agentic AI. autonomous, multi-step workflows with orchestration, tool use, parallel agents, data movement, and enterprise integration has dramatically increased the importance of strong host CPUs alongside GPUs. This shifts the CPU-to-GPU ratio higher and makes balanced systems critical toward 1:1 to 5:1 as enterprises testing more than 5-10 agents. AMD EPYC Venice excels ~Leadership core density (up to 256 Zen 6 cores per socket) for running many agents in parallel, orchestration layers, and high-throughput control-plane tasks. ~Superior performance-per-core and power efficiency ( up to 2.1x higher perf/core and 2.26x better SPECpower vs. NVIDIA Grace in benchmarks). ~Tight integration in Helios: One Venice CPU + multiple MI450 GPUs per node, enabling efficient data feeding to GPUs ("zero-copy"), parallel execution, and full rack utilization for complex agentic loops. Hyperscalers (Meta, Microsoft, Amazon, Google, Softbank) and AI natives (OpenAI, Anthropic...) are adopting high-core EPYC at scale specifically for these agentic demands, as CPUs now handle a larger share of non-model work (orchestration, policy enforcement, tool calls). This complements AMD’s lower-cost GPUs for overall TCO wins. ~Agents often generate 10–100x+ more tokens per task due to iterative reasoning chains, multiple tool calls, verification loops, and long-context orchestration. ~Goldman Sachs forecasts token consumption multiplying 24x by 2030 (to 120 quadrillion tokens/month) largely driven by agentic adoption in consumer and enterprise. ~Enterprise data shows agent-pattern workloads growing at 680% annualized rates, projected to surpass conversational AI in token volume by Q3 2026. ~Daily enterprise agent token consumption is already in the billions, with complex workflows (coding, workflows, analysis) amplifying this dramatically. 4. Competitive Edge: Winning Customers from Anthropic Anthropic’s Claude models (especially Opus/Sonnet) excel in complex reasoning and agentic coding, commanding premium positioning. However, their higher underlying costs (heavier reliance on third-party cloud with margins) limit pricing flexibility compared to OpenAI’s owned Helios capacity. Anthropic is on track to generate $10.9 billion in Q2 revenue. The company expects to achieve its first-ever quarterly adjusted operating profit of $559 million. However, sustaining full-year profitability remains challenging due to immense computing and model training costs The truth is, Anthropic has no choice but to buy as much $AMD chips as possible if they want to compete with OpenAI or get investors attention. This 5% adjusted operating profit to revenue ratio is just pathetic. Current pricing dynamics (2026): OpenAI already undercuts on many tiers ( flagship output tokens significantly cheaper than equivalent Claude Opus). Nano/mini models offer 5–10x advantages for volume work. Anthropic holds edges in long-context flat pricing and certain reasoning quality. OpenAI after Helios Rack Ownership, At $0.0003–$0.0005/M effective costs, OpenAI gains massive headroom to: ~Aggressively discount high-volume agentic tiers or bundles. ~Offer “unlimited” enterprise plans or usage-based models that Anthropic struggles to match without margin erosion. ~Target cost-sensitive, high-throughput agent deployments (dev tools, automation platforms) where token bills explode. Enterprises facing $ millions in monthly agentic bills will migrate to the provider delivering better economics at scale. OpenAI’s combination of strong models (o-series reasoning) + lowest TCO positions it to erode Anthropic’s enterprise share, especially as agentic becomes the dominant token consumer. Cheaper tokens expand the total addressable market dramatically. This feeds the data/model improvement loop, justifying further capex. AMD benefits from proven scale pulling in more customers (Meta, Oracle, Microsfot, Amazon, Softbank, TensorWave, LumaAI ... already aligned on Helios). Conclusion: Dr. Lisa Su has been laser focused on inference economics since at least 2022–2023, repeatedly emphasizing that the real battleground for AI scalability would be TCO, power efficiency (TDP), and ultimately tokens per dollar and per watt not just raw training FLOPS. While many viewed inference as a secondary, commoditized workload, Dr. Su architected AMD’s roadmap around rack-scale systems optimized for high-volume, sustained inference that would dominate as models matured and usage exploded. Helios represents the culmination of that multi-year bet: a fully integrated, open platform designed precisely for the economics of massive token throughput. This deep, strategic partnership with OpenAI starting with the 1GW Helios deployment in H2 2026 and scaling to 6GW, is the embodiment of that shared vision. Both companies foresaw a future where agentic AI models evolve to become extraordinarily token-hungry: autonomous agents executing complex, iterative workflows with planning, tool use, verification loops, and long-context reasoning. These workloads can consume 100x+ more tokens per task than traditional chat or single-turn generation, driving exponential demand as capabilities improve and enterprises deploy them at scale. By owning and optimizing this massive Helios capacity at GW scale, OpenAI achieves inference costs as low as $0.0003–$0.0005 per million tokens. This structural cost advantage allows OpenAI to absorb the coming token explosion profitably, dramatically lower effective pricing for enterprises, and win high-volume agentic workloads from higher-cost competitors like Anthropic. What was once a prohibitive monthly token bill becomes an affordable accelerator for productivity and innovation. The OpenAI-AMD alliance validates Dr. Su’s prescient strategy and turns the Agentic flywheel into reality: Collapsing inference costs → explosive token consumption → richer data and better models → accelerate greater demand. This partnership doesn’t just address today’s economics, it positions both leaders at the center of the infrastructure buildout that will power AI’s next decade. By delivering the lowest inference economics at scale, OpenAI not only solves enterprise bill pain but gains a decisive weapon to win share from higher-cost rivals like Anthropic. And that is why OpenAI and $META will deploy EPYC Dense Rack Not Financial Advice! DYOR! Research Purpose Only!

Mike

84,951 次观看 • 2 个月前

Google just confirmed the first case of hackers using AI to build a zero-day exploit from scratch. An actual zero-day vulnerability that no human had EVER found before, discovered by an AI model, turned into a working weapon, and aimed at a mass exploitation campaign targeting thousands of systems simultaneously. Google's Threat Intelligence Group caught it yesterday and killed the operation before it scaled. But the details of how it worked are genuinely scary: The AI found a flaw in a popular two-factor authentication system that traditional security tools had missed entirely. The vulnerability was a logic error buried deep in the authentication flow where a developer had hard-coded a trust exception years ago. No human security researcher or automated scanner had caught it. The flaw was invisible to EVERY tool the cybersecurity industry has built over the past two decades. But the AI spotted it immediately. Then it wrote a full Python exploit script to weaponize it. Google's analysts could tell the code was AI-generated because it had textbook formatting, educational comments explaining every function, and even a hallucinated severity score that doesn't exist in any real database. The AI literally graded its own attack with a fake rating. So the code had MISTAKES in it. The criminals' implementation was clumsy enough that it probably interfered with the actual deployment. This was the sloppy first attempt by people who are still learning how to use these tools. And it still found a vulnerability that the entire cybersecurity industry missed. Google's chief threat analyst John Hultquist said: "There's a misconception that the AI vulnerability race is imminent. The reality is that it's already begun. For every zero-day we can trace back to AI, there are probably many more out there." But here's where it gets truly insane... This wasn't even a sophisticated operation. North Korea's APT45 hacking unit is sending thousands of repetitive prompts to AI models, recursively analyzing known vulnerabilities and building an entire exploit arsenal that would be physically impossible for human hackers to assemble at the same speed. They're essentially industrializing cyberattacks. A Chinese state-linked group jailbroke Google's own Gemini by simply asking it to "pretend to be a network security expert" and then used that persona to research how to hack TP-Link routers and corporate file transfer systems. Another Chinese group deployed autonomous AI agents that probed a Japanese tech firm with minimal human oversight, deciding on their own which tools to use and pivoting between targets based on internal reasoning. And then there's PROMPTSPY, an Android backdoor that calls Google's Gemini API to read your phone screen in real time, navigate your interface autonomously, capture your biometric data, replay your lock screen PIN, and block you from uninstalling it by placing an invisible overlay over the uninstall button. It literally OPERATES your phone using commercial AI tools anyone can access. Everyone spent the last 3 years arguing about whether AI would take people's jobs. Meanwhile AI is making every password, every firewall, and every two-factor authentication system on Earth fundamentally less secure. The entire $190 billion cybersecurity industry was built on one assumption: that finding vulnerabilities is hard and requires deep expertise. But AI just removed that assumption from the equation. And the scariest part is that Google said the criminals made errors this time. The implementation was rough and the campaign probably didn't fully work. These were amateurs, now imagine what professionals are able to do. There's a reason Sam Altman predicted an inevitable massive cyberattack THIS year. What do you think?
1:45

Google just confirmed the first case of hackers using AI to build a zero-day exploit from scratch. An actual zero-day vulnerability that no human had EVER found before, discovered by an AI model, turned into a working weapon, and aimed at a mass exploitation campaign targeting thousands of systems simultaneously. Google's Threat Intelligence Group caught it yesterday and killed the operation before it scaled. But the details of how it worked are genuinely scary: The AI found a flaw in a popular two-factor authentication system that traditional security tools had missed entirely. The vulnerability was a logic error buried deep in the authentication flow where a developer had hard-coded a trust exception years ago. No human security researcher or automated scanner had caught it. The flaw was invisible to EVERY tool the cybersecurity industry has built over the past two decades. But the AI spotted it immediately. Then it wrote a full Python exploit script to weaponize it. Google's analysts could tell the code was AI-generated because it had textbook formatting, educational comments explaining every function, and even a hallucinated severity score that doesn't exist in any real database. The AI literally graded its own attack with a fake rating. So the code had MISTAKES in it. The criminals' implementation was clumsy enough that it probably interfered with the actual deployment. This was the sloppy first attempt by people who are still learning how to use these tools. And it still found a vulnerability that the entire cybersecurity industry missed. Google's chief threat analyst John Hultquist said: "There's a misconception that the AI vulnerability race is imminent. The reality is that it's already begun. For every zero-day we can trace back to AI, there are probably many more out there." But here's where it gets truly insane... This wasn't even a sophisticated operation. North Korea's APT45 hacking unit is sending thousands of repetitive prompts to AI models, recursively analyzing known vulnerabilities and building an entire exploit arsenal that would be physically impossible for human hackers to assemble at the same speed. They're essentially industrializing cyberattacks. A Chinese state-linked group jailbroke Google's own Gemini by simply asking it to "pretend to be a network security expert" and then used that persona to research how to hack TP-Link routers and corporate file transfer systems. Another Chinese group deployed autonomous AI agents that probed a Japanese tech firm with minimal human oversight, deciding on their own which tools to use and pivoting between targets based on internal reasoning. And then there's PROMPTSPY, an Android backdoor that calls Google's Gemini API to read your phone screen in real time, navigate your interface autonomously, capture your biometric data, replay your lock screen PIN, and block you from uninstalling it by placing an invisible overlay over the uninstall button. It literally OPERATES your phone using commercial AI tools anyone can access. Everyone spent the last 3 years arguing about whether AI would take people's jobs. Meanwhile AI is making every password, every firewall, and every two-factor authentication system on Earth fundamentally less secure. The entire $190 billion cybersecurity industry was built on one assumption: that finding vulnerabilities is hard and requires deep expertise. But AI just removed that assumption from the equation. And the scariest part is that Google said the criminals made errors this time. The implementation was rough and the campaign probably didn't fully work. These were amateurs, now imagine what professionals are able to do. There's a reason Sam Altman predicted an inevitable massive cyberattack THIS year. What do you think?

Ricardo

50,564 次观看 • 3 个月前

Dear ICP community, the Internet Computer has now been running strong for 5 years 👏👏👏 Here is a celebratory preview of ICP "cloud engines," the sovereign frontier cloud technology the network shall soon provide from Main points: — Cloud engines enable anyone to spin up their own sovereign frontier cloud. The technology involves an extraordinary inventive step, in which cloud is created from a mathematically secure network of nodes. The nodes run as part of the Internet Computer network ( but are selected and configured by the cloud engine's owner. — The frontier cloud provided by engines is strongly focused on enabling AI agents to build and update online applications and services for us. The world is changing fast, and nearly all new online apps and services are already being built with the help of AI, and thus cloud engines target the future of cloud. — Software hosted on cloud engines is tamperproof, which means that it is immune to infrastructure hacks, because it runs inside a mathematically secure network protocol, rather than on computers directly. This means that AI agents, and those building with them, don't need to have a security team in the loop, or to trust someone else's security team. This is crucial, because in the future, non technical people will demand the freedom to build with full automation — where they just need to issue instructions to AI about what to build, and don't need to worry about anything or anyone else. Of course, apps and services running on engines are also vastly safer from the new breed of hacker being enabled by frontier AI. (The cloud engines themselves are also "tamperproof." Even if a hacker gains physical access to some portion of a cloud engine's nodes, and can make arbitrary changes, the computations and data of the hosted apps and services cannot be corrupted or interrupted so long as the network's fault bounds aren't exceeded. The recent hack of Vercel, a major cloud platform, which gave hackers access to the apps it hosted, provides additional perspective on the importance of this advantage.) — Software hosted on cloud engines is guaranteed to run, so long as a sufficient number of the engine's nodes are running. This means that AI can build applications and services without the need to have a human systems admin team constantly tinkering with the underlying platform to keep it running, which is again crucial, because in the future, non technical people will expect the freedom to use AI to build without the support of others. — New frontier programming language technology, in the form of the Motoko language developed by Caffeine Labs, leverages seminal "orthogonal persistence" technology that unifies program logic and data to deliver further unlocks for AI (Motoko is the first computer language being developed that targets agents that are writing software rather than humans engineers per se). Nowadays, AI can build and update production apps at a prodigious rate, even at the speed of conversation. But it can also make mistakes, and there's a risk that an update it creates might be "lossy" in the sense it causes some transformed data to be lost. Again, in this new world, it's both undesirable and impractical for everyone to have to have a systems admin team on-hand to detect lossy updates and roll them back, but Motoko provides a solution: it can detect new software updates are lossy before they are applied, reducing potentially catastrophic errors by AI to harmless coding retries. — Software hosted on cloud engines is "serverless" but unlike traditional serverless software, directly it directly incorporates data through "orthogonal persistence." Another key purpose is simplify backend software logic and fuel the modeling power of AI by increasing abstraction (sorry for the technical language!!!). Put simply, this enables AI to produce more sophisticated backends, faster, and at dramatically lower costs, as measured by the number AI API tokens consumed during coding. (Tip for the technical: orthogonal persistence is a new paradigm where "the program is the database," and data lives inside program variables, which is possible because it's as if hosted software runs forever in persistent memory). — An expanding database of skills at shall make it possible to develop and directly deploy apps and services to your cloud engines directly from Claude Code, Perplexity, Codex and other AI platforms. Further, your account on can be connected, so that new apps and updates created through conversation automatically appear hosted from your cloud engine. In the future, R&D is going to be very seamless. You converse with AI, and your secure and unstoppable apps or services are created or updated. Cloud engines are designed to directly support this "self-writing cloud" future where we can work hands-free. — Tech sovereignty is becoming a huge issue worldwide, with governments and corporations seeking to create sovereign tech stacks owing to geopolitical tensions. Increasingly, people are realizing that tech provided by foreign nations can come with hidden backdoors and kills switches, from the base platform, right up through hosted apps and services. ICP technology is open source, and those building on ICP using AI own their own source code. When you have the source code, you can verify that there are no backdoors, and when you own the source code thanks to AI, you can update it at will, freeing you from vendor lock-in. But cloud engines take sovereignty much further... — You create a cloud engine by selecting the nodes that will be combined. You can choose the class of nodes used, and their number, but more importantly, you can choose who operates the nodes, and where they are located. Almost any configuration is possible, because the Internet Computer scales the security privileges afforded to hosted software within the network according to configuration (software hosted on cloud engines can directly interoperate with software on other engines and traditional subnets, but base restrictions are applied according to security rules). A cloud engine can be created within a region such as Europe, to comply with regs such as GDPR, or completely within a sovereign state like Switzerland or Pakistan. But cloud engines go further still... — Sovereignty is also about freedom from vendor lock-in. Cloud engines are essentially ICP (Internet Computer Protocol) network configurations, and this means the underlying compute nodes they combine can be swapped out without interrupting their hosted apps and services. This is a big deal. In addition, cloud engines now support nodes that are instances running on Big Tech's clouds, in addition to nodes that are dedicated specialized hardware, as per the Gen I and Gen II nodes that dominate the Internet Computer today. For example, it is possible to have an engine running across different AWS data centers, say, and then reconfigure the engine to run across a mixture of AWS, Google, Azure and Hetzner for even more resilience, without the users of hosted apps and services noticing a thing. That's true freedom. — Sovereign AI is becoming increasingly important too, and cloud engines allow special "AI nodes" to be added to them, so that hosted software can perform inference on hardware provisioned by the owner from a location the owner has selected. Even though the AI nodes are only accessible within the cloud engine, they can still benefit from the forthcoming Internet Intelligence Gateway (IG), which will make it possible to validate inference performed on key frontier open weights LLMs, even when the inference is performed on completely independent AI clouds. When the results of inference are received, this technology can verify that neither the prompt+context (input) nor the inference result (output) have been modified, and that the results were produced by the precise LLM expected. This ensures that AI clouds don't cheat by running inference on cheaper models than are being paid for, and bad actors aren't modifying the inputs or outputs to surreptitiously insert advertising into results, say, or change facts, or insert malware when code is being generated. What's super cool about this technology is the cost of the verification is scalable. A very valuable additional security can be achieved with only 1-2% of extra cost. — Scaling apps and services when they hit capacity limits is another thorny problem that cloud engines help the world address. Engines make scaling possible without rewriting or reconfiguring software. The query workload capacity of hosted software can be horizontally scaled simply by adding new nodes to an engine, and nodes can also be added in geographical proximity to demand. Meanwhile, update workload capacity can first be scaled-up by swapping an engine's nodes out for the next class up, and then when no larger class of node is available, horizontally scaled-out by "splitting" the engine into two, which doubles available capacity. (Technical tip: horizontally scaling update capacity by splitting engines requires multi-canister architectures). — For those who have been following how Caffeine builds apps that can efficiently store large numbers of files, I should mention that apps built on cloud engines will also support the new ICP Blob Storage cloud network (since cloud engines currently have up to about 3 TB of memory, which apps storing large amounts of files can easily exceed). We are also working on allowing blob storage nodes to be added to cloud engines, to enable sovereign mass blob storage within an engine, similarly to how AI nodes can be added currently. — Lastly, but certainly not least, I should mention that cloud engines are multi-blockchain capable, and ready for digital assets, thanks to the clever math at their core. For example, an e-commerce service built on a cloud engine can securely accept and custody stablecoin payments, or a multi-chain DEX could be hosted. Further, engines can support software autonomy (software orchestrated and controlled by other autonomous software, in a decentralized way) and can themselves be orchestrated by SNS technology, and thus run autonomously too. Today, though, the focus is on *mainstream* cloud. This year, the cloud industry will generate approximately one trillion dollars in revenue. That number is already huge, but is expected to grow to two trillion dollars by 2030. After years of continuous development, which have seen more than $500m spent on R&D, the Internet Computer network is now tacking directly toward this mainstream cloud market with cloud engine technology. In their first version, cloud engines are not meant to be a cloud panacea. For example, currently they are not ideal for working with big data. You should use something like DataBricks for that. Cloud engines are carefully targeted at enabling AI to produce traditional online applications and services, including SaaS, in a safer and more productive way, which represents a new market segment with tremendous potential. Of course, DFINITY will continue to work relentlessly to push forward ICP's capabilities, so expect further developments. It's worth mentioning that this cloud segment isn't just about creating new apps and services using AI, it's also about replacing legacy systems and apps built on super expensive SaaS services. Caffeine Labs is working to produce technology (Caffeine Snorkel) that can study an enterprise's legacy systems and app built on SaaS, create replacement systems and apps, and migrate the data, while supporting key stakeholders through the process over email and chat, with full automation. Thus the legacy systems and SaaS markets shall also be addressed by cloud engines. Zooming out, and reasoning in a more metaphysical way, we believe, as we always have, that there is room for a new kind of cloud created by mathematical networks, that provides seminal advances in the fields of security and resilience, as well as true sovereignty and freedom from lock-in. That this same technology, with the help of additional technologies like orthogonal persistence and Motoko, enables AI to build for us without the need for so much oversight, and to create more backend sophistication while consuming fewer AI API tokens, enables ICP to bring game-changing advances to the world. Cloud engines will work synergistically with the Intelligence Gateway, which will enable apps and services running on engines to seamlessly leverage AI, wherever that AI is running, while providing verifiability at extremely low cost for open weights frontier models. We believe that cloud engines represent an inflection point in the storied history of the Internet Computer project, and I'm very proud to be sharing the details with you on the network's fifth birthday 💪 I'll be back with more news soon!!
19:06

Dear ICP community, the Internet Computer has now been running strong for 5 years 👏👏👏 Here is a celebratory preview of ICP "cloud engines," the sovereign frontier cloud technology the network shall soon provide from Main points: — Cloud engines enable anyone to spin up their own sovereign frontier cloud. The technology involves an extraordinary inventive step, in which cloud is created from a mathematically secure network of nodes. The nodes run as part of the Internet Computer network ( but are selected and configured by the cloud engine's owner. — The frontier cloud provided by engines is strongly focused on enabling AI agents to build and update online applications and services for us. The world is changing fast, and nearly all new online apps and services are already being built with the help of AI, and thus cloud engines target the future of cloud. — Software hosted on cloud engines is tamperproof, which means that it is immune to infrastructure hacks, because it runs inside a mathematically secure network protocol, rather than on computers directly. This means that AI agents, and those building with them, don't need to have a security team in the loop, or to trust someone else's security team. This is crucial, because in the future, non technical people will demand the freedom to build with full automation — where they just need to issue instructions to AI about what to build, and don't need to worry about anything or anyone else. Of course, apps and services running on engines are also vastly safer from the new breed of hacker being enabled by frontier AI. (The cloud engines themselves are also "tamperproof." Even if a hacker gains physical access to some portion of a cloud engine's nodes, and can make arbitrary changes, the computations and data of the hosted apps and services cannot be corrupted or interrupted so long as the network's fault bounds aren't exceeded. The recent hack of Vercel, a major cloud platform, which gave hackers access to the apps it hosted, provides additional perspective on the importance of this advantage.) — Software hosted on cloud engines is guaranteed to run, so long as a sufficient number of the engine's nodes are running. This means that AI can build applications and services without the need to have a human systems admin team constantly tinkering with the underlying platform to keep it running, which is again crucial, because in the future, non technical people will expect the freedom to use AI to build without the support of others. — New frontier programming language technology, in the form of the Motoko language developed by Caffeine Labs, leverages seminal "orthogonal persistence" technology that unifies program logic and data to deliver further unlocks for AI (Motoko is the first computer language being developed that targets agents that are writing software rather than humans engineers per se). Nowadays, AI can build and update production apps at a prodigious rate, even at the speed of conversation. But it can also make mistakes, and there's a risk that an update it creates might be "lossy" in the sense it causes some transformed data to be lost. Again, in this new world, it's both undesirable and impractical for everyone to have to have a systems admin team on-hand to detect lossy updates and roll them back, but Motoko provides a solution: it can detect new software updates are lossy before they are applied, reducing potentially catastrophic errors by AI to harmless coding retries. — Software hosted on cloud engines is "serverless" but unlike traditional serverless software, directly it directly incorporates data through "orthogonal persistence." Another key purpose is simplify backend software logic and fuel the modeling power of AI by increasing abstraction (sorry for the technical language!!!). Put simply, this enables AI to produce more sophisticated backends, faster, and at dramatically lower costs, as measured by the number AI API tokens consumed during coding. (Tip for the technical: orthogonal persistence is a new paradigm where "the program is the database," and data lives inside program variables, which is possible because it's as if hosted software runs forever in persistent memory). — An expanding database of skills at shall make it possible to develop and directly deploy apps and services to your cloud engines directly from Claude Code, Perplexity, Codex and other AI platforms. Further, your account on can be connected, so that new apps and updates created through conversation automatically appear hosted from your cloud engine. In the future, R&D is going to be very seamless. You converse with AI, and your secure and unstoppable apps or services are created or updated. Cloud engines are designed to directly support this "self-writing cloud" future where we can work hands-free. — Tech sovereignty is becoming a huge issue worldwide, with governments and corporations seeking to create sovereign tech stacks owing to geopolitical tensions. Increasingly, people are realizing that tech provided by foreign nations can come with hidden backdoors and kills switches, from the base platform, right up through hosted apps and services. ICP technology is open source, and those building on ICP using AI own their own source code. When you have the source code, you can verify that there are no backdoors, and when you own the source code thanks to AI, you can update it at will, freeing you from vendor lock-in. But cloud engines take sovereignty much further... — You create a cloud engine by selecting the nodes that will be combined. You can choose the class of nodes used, and their number, but more importantly, you can choose who operates the nodes, and where they are located. Almost any configuration is possible, because the Internet Computer scales the security privileges afforded to hosted software within the network according to configuration (software hosted on cloud engines can directly interoperate with software on other engines and traditional subnets, but base restrictions are applied according to security rules). A cloud engine can be created within a region such as Europe, to comply with regs such as GDPR, or completely within a sovereign state like Switzerland or Pakistan. But cloud engines go further still... — Sovereignty is also about freedom from vendor lock-in. Cloud engines are essentially ICP (Internet Computer Protocol) network configurations, and this means the underlying compute nodes they combine can be swapped out without interrupting their hosted apps and services. This is a big deal. In addition, cloud engines now support nodes that are instances running on Big Tech's clouds, in addition to nodes that are dedicated specialized hardware, as per the Gen I and Gen II nodes that dominate the Internet Computer today. For example, it is possible to have an engine running across different AWS data centers, say, and then reconfigure the engine to run across a mixture of AWS, Google, Azure and Hetzner for even more resilience, without the users of hosted apps and services noticing a thing. That's true freedom. — Sovereign AI is becoming increasingly important too, and cloud engines allow special "AI nodes" to be added to them, so that hosted software can perform inference on hardware provisioned by the owner from a location the owner has selected. Even though the AI nodes are only accessible within the cloud engine, they can still benefit from the forthcoming Internet Intelligence Gateway (IG), which will make it possible to validate inference performed on key frontier open weights LLMs, even when the inference is performed on completely independent AI clouds. When the results of inference are received, this technology can verify that neither the prompt+context (input) nor the inference result (output) have been modified, and that the results were produced by the precise LLM expected. This ensures that AI clouds don't cheat by running inference on cheaper models than are being paid for, and bad actors aren't modifying the inputs or outputs to surreptitiously insert advertising into results, say, or change facts, or insert malware when code is being generated. What's super cool about this technology is the cost of the verification is scalable. A very valuable additional security can be achieved with only 1-2% of extra cost. — Scaling apps and services when they hit capacity limits is another thorny problem that cloud engines help the world address. Engines make scaling possible without rewriting or reconfiguring software. The query workload capacity of hosted software can be horizontally scaled simply by adding new nodes to an engine, and nodes can also be added in geographical proximity to demand. Meanwhile, update workload capacity can first be scaled-up by swapping an engine's nodes out for the next class up, and then when no larger class of node is available, horizontally scaled-out by "splitting" the engine into two, which doubles available capacity. (Technical tip: horizontally scaling update capacity by splitting engines requires multi-canister architectures). — For those who have been following how Caffeine builds apps that can efficiently store large numbers of files, I should mention that apps built on cloud engines will also support the new ICP Blob Storage cloud network (since cloud engines currently have up to about 3 TB of memory, which apps storing large amounts of files can easily exceed). We are also working on allowing blob storage nodes to be added to cloud engines, to enable sovereign mass blob storage within an engine, similarly to how AI nodes can be added currently. — Lastly, but certainly not least, I should mention that cloud engines are multi-blockchain capable, and ready for digital assets, thanks to the clever math at their core. For example, an e-commerce service built on a cloud engine can securely accept and custody stablecoin payments, or a multi-chain DEX could be hosted. Further, engines can support software autonomy (software orchestrated and controlled by other autonomous software, in a decentralized way) and can themselves be orchestrated by SNS technology, and thus run autonomously too. Today, though, the focus is on *mainstream* cloud. This year, the cloud industry will generate approximately one trillion dollars in revenue. That number is already huge, but is expected to grow to two trillion dollars by 2030. After years of continuous development, which have seen more than $500m spent on R&D, the Internet Computer network is now tacking directly toward this mainstream cloud market with cloud engine technology. In their first version, cloud engines are not meant to be a cloud panacea. For example, currently they are not ideal for working with big data. You should use something like DataBricks for that. Cloud engines are carefully targeted at enabling AI to produce traditional online applications and services, including SaaS, in a safer and more productive way, which represents a new market segment with tremendous potential. Of course, DFINITY will continue to work relentlessly to push forward ICP's capabilities, so expect further developments. It's worth mentioning that this cloud segment isn't just about creating new apps and services using AI, it's also about replacing legacy systems and apps built on super expensive SaaS services. Caffeine Labs is working to produce technology (Caffeine Snorkel) that can study an enterprise's legacy systems and app built on SaaS, create replacement systems and apps, and migrate the data, while supporting key stakeholders through the process over email and chat, with full automation. Thus the legacy systems and SaaS markets shall also be addressed by cloud engines. Zooming out, and reasoning in a more metaphysical way, we believe, as we always have, that there is room for a new kind of cloud created by mathematical networks, that provides seminal advances in the fields of security and resilience, as well as true sovereignty and freedom from lock-in. That this same technology, with the help of additional technologies like orthogonal persistence and Motoko, enables AI to build for us without the need for so much oversight, and to create more backend sophistication while consuming fewer AI API tokens, enables ICP to bring game-changing advances to the world. Cloud engines will work synergistically with the Intelligence Gateway, which will enable apps and services running on engines to seamlessly leverage AI, wherever that AI is running, while providing verifiability at extremely low cost for open weights frontier models. We believe that cloud engines represent an inflection point in the storied history of the Internet Computer project, and I'm very proud to be sharing the details with you on the network's fifth birthday 💪 I'll be back with more news soon!!

dom | icp

293,234 次观看 • 3 个月前

New video from Shreya Shankar on data processing with LLMs at scale, an underrated topic! Shreya starts with a real use case: public defenders analyzing case files for racial bias (4:08). Hundreds of pages per defendant. Court transcripts, police reports, news articles. Running GPT-5 on everything costs a fortune. Her solution: treat LLMs like database operators. Semantic Map, Filter, Reduce (9:18). Databricks, BigQuery, and Snowflake are already shipping this as "AI SQL." She discusses how starting at 12:51: a query optimizer for LLMs. Traditional databases rewrite queries for efficiency. Shreya does the same for LLM pipelines (semantic versions of split, map, reduce that are LLM specific, along with query decomposition). For example, trivial LLM calls are replaced with Python functions. These "rewrite directives" improve both cost AND accuracy. She also talks about a cost optimization technique: Task Cascades (30:00). Instead of running GPT-5 on every document, first ask cheap questions. "Is there any lower court mentioned?" If no, the document clearly doesn't overturn a lower court. There are many other routing questions you can ask to reduce the amount of text sent to the LLM. This requires careful optimization and tuning to get right. She explains how to do this in the video. She runs through a production example that achieved 86% cost reduction while retaining 90% accuracy. --- At 41:26, Shreya shifts to HCI. She built DocWrangler, an IDE for LLM pipelines. The design is based on "Three Gulfs" (44:35): 1. Comprehension: You don't know what's in your data 2. Specification: "Only prescription meds" is hard to operationalize 3. Generalization: A prompt that works on 10 examples fails at 10,000 Users invented "throwaway pipelines" just to explore their data before doing real analysis. Pipelines with no analytical purpose: "summarize these documents," "extract key ideas." Just ways to learn what's in their data before doing the real work. DocWrangler makes this a first-class feature. --- In the last bit of the video Shreya discusses why you can't know what "good" means until you see examples. In one study, a medical analyst extracted medications from doctor-patient transcripts. As they inspected outputs, they noticed every medication appeared with a dosage. They hadn't anticipated this. Now they wanted dosages too. They also saw Tylenol and ibuprofen appearing and realized: "Actually, I only want prescription medications." Shreya calls this "criteria drift." Your evaluation criteria evolve as you see more outputs. This matters because standard ML assumes fixed metrics: define them upfront, collect labels, measure. But with LLMs on fuzzy tasks, that assumption breaks. You discover what you actually want through the process of evaluating. If you don't account for criteria drift, you end up optimizing for a stale rubric. DocWrangler and EvalGen accommodate this by placing the human in the loop thoughtfully. Chapter timestamps: (4:08) - The problem: unstructured data at scale (9:18) - Semantic operators (Map, Filter, Reduce) (12:51) - Query optimization for LLMs (18:15) - Data decomposition (chunking) (30:00) - Task Cascades (86% cost reduction) (41:26) - DocWrangler IDE (44:35) - Three Gulfs framework (51:50) - Evaluation criteria drift More links in reply
1:14:16

New video from Shreya Shankar on data processing with LLMs at scale, an underrated topic! Shreya starts with a real use case: public defenders analyzing case files for racial bias (4:08). Hundreds of pages per defendant. Court transcripts, police reports, news articles. Running GPT-5 on everything costs a fortune. Her solution: treat LLMs like database operators. Semantic Map, Filter, Reduce (9:18). Databricks, BigQuery, and Snowflake are already shipping this as "AI SQL." She discusses how starting at 12:51: a query optimizer for LLMs. Traditional databases rewrite queries for efficiency. Shreya does the same for LLM pipelines (semantic versions of split, map, reduce that are LLM specific, along with query decomposition). For example, trivial LLM calls are replaced with Python functions. These "rewrite directives" improve both cost AND accuracy. She also talks about a cost optimization technique: Task Cascades (30:00). Instead of running GPT-5 on every document, first ask cheap questions. "Is there any lower court mentioned?" If no, the document clearly doesn't overturn a lower court. There are many other routing questions you can ask to reduce the amount of text sent to the LLM. This requires careful optimization and tuning to get right. She explains how to do this in the video. She runs through a production example that achieved 86% cost reduction while retaining 90% accuracy. --- At 41:26, Shreya shifts to HCI. She built DocWrangler, an IDE for LLM pipelines. The design is based on "Three Gulfs" (44:35): 1. Comprehension: You don't know what's in your data 2. Specification: "Only prescription meds" is hard to operationalize 3. Generalization: A prompt that works on 10 examples fails at 10,000 Users invented "throwaway pipelines" just to explore their data before doing real analysis. Pipelines with no analytical purpose: "summarize these documents," "extract key ideas." Just ways to learn what's in their data before doing the real work. DocWrangler makes this a first-class feature. --- In the last bit of the video Shreya discusses why you can't know what "good" means until you see examples. In one study, a medical analyst extracted medications from doctor-patient transcripts. As they inspected outputs, they noticed every medication appeared with a dosage. They hadn't anticipated this. Now they wanted dosages too. They also saw Tylenol and ibuprofen appearing and realized: "Actually, I only want prescription medications." Shreya calls this "criteria drift." Your evaluation criteria evolve as you see more outputs. This matters because standard ML assumes fixed metrics: define them upfront, collect labels, measure. But with LLMs on fuzzy tasks, that assumption breaks. You discover what you actually want through the process of evaluating. If you don't account for criteria drift, you end up optimizing for a stale rubric. DocWrangler and EvalGen accommodate this by placing the human in the loop thoughtfully. Chapter timestamps: (4:08) - The problem: unstructured data at scale (9:18) - Semantic operators (Map, Filter, Reduce) (12:51) - Query optimization for LLMs (18:15) - Data decomposition (chunking) (30:00) - Task Cascades (86% cost reduction) (41:26) - DocWrangler IDE (44:35) - Three Gulfs framework (51:50) - Evaluation criteria drift More links in reply

Hamel Husain

35,619 次观看 • 8 个月前

$AMD $620/share is too conservative for 2026 🧵 Some quick facts before I dive into this super long thread: $META allocated 42% GPUs to $AMD and 58% to $NVDA OpenAI allocated 6GW(38%) to $AMD and 10GW to $NVDA My $620 PT below by end of 2026 was only for 10-15% market share. I believe $AMD is going to have much much higher market share than I projected. The AI accelerator market is exploding, projected to reach $500 billion by 2028(is now heading $1Tril), driven by insatiable demand for training and inference compute in large language models (LLMs), recommendation systems, and autonomous systems. Nvidia ($NVDA) has long held a stranglehold, commanding over 90% market share through its CUDA ecosystem and superior rack-scale solutions. However, AMD is mounting a formidable challenge, leveraging cost advantages, open-source software momentum, and hyperscaler partnerships to erode Nvidia's moat. Recent deals—such as Meta's ($META) allocation of 42% of its GPU capacity to AMD and OpenAI's commitment to 6GW of AMD compute (versus 10GW for Nvidia)—signal a tipping point. At the forefront is AMD's Instinct MI450 series, a next-generation AI GPU slated for H2 2026 launch, which promises "no-excuses" leadership in training, inference, and distributed workloads. This analysis dissects how AMD will capture more market share and why hyperscalers like $Meta , xAI , Oracle , and others are poised to become voracious buyers of the MI450. AMD's AI GPU revenue has surged from negligible levels in 2022 to an estimated $4-5 billion in 2025, capturing ~6% of the data center GPU market. This growth stems from the Instinct MI300X, which offers 141GB of HBM3 memory and competitive FP8/FP16 performance at 20-30% lower cost than Nvidia's H100. Hyperscalers, facing NVIDIA 's overcharging, have turned to AMD for diversification. Meta, for instance, plans 600,000 H100-equivalent GPUs by end-2024, with ~42% (or 250,000+ units) sourced from AMD's MI300 series for inference tasks like image editing and AI assistants. Similarly, OpenAI's recent multi-year deal commits to 6GW of AMD compute—equivalent to ~300,000-400,000 MI450 GPUs—starting with 1GW in 2026, explicitly to counterbalance its 10GW Nvidia allocation. These aren't one-offs. Microsoft Azure, Amazon AWS, and Oracle Cloud Infrastructure (OCI) have integrated MI300X for AI workloads, with Oracle deploying 30,000 MI355X units in zettascale clusters. xAI, Elon Musk Musk's AI venture, ran 30% of Grok-1's production traffic on MI300X GPUs and has confirmed ongoing purchases. Collectively, these partners represent over $400 billion in projected AI infrastructure spend through 2028, with AMD targeting up to 40% market share. For those that subscribed, I wrote a specific thread on how AMD "secret weapon" is going to change the game in 2026 with an improved designs on all its products, yes AMD has patent on it. Software is the linchpin. AMD's ROCm platform, once derided as "half-baked," now supports day-zero integration for Llama-4, DeepSeek V3, and GPT-OSS models—closing the CUDA gap. Benchmarks show MI355X (MI450 precursor) outperforming Nvidia's B200 in inference by 1.5-2x on memory-bound tasks, at 25-35% lower TCO. For training, MI450's rack-scale IF128 configuration (128 GPUs, 1.4 PB/s intra-rack bandwidth) rivals Nvidia's VR200 NVL144, enabling clusters like xAI's Colossus (scaling to 1M GPUs). My below thread projected Etimated conservative FY 25 revenue: $34-$36B Estimated conservative FY 26 revenue: $55B-$62B Below is why $AMD is revenue is going to be much higher after OpenAI deal. 1. OpenAI 1GW in 2026. With high demand for MI355X at $30,000k+ per unit, with MI450 is likely to be sold in the $45k-$55k. We can safely calcuate 1GW would require roughly 400,000 MI450 GPUs. or Roughly ~$20B revenue in 2026 alone from OpenAI. That would mean $AMD would hit $56B just from one partnership(OpenAI) in 2026 2. $META, the biggest spender on AI Infrastructure right now, Daddy Zuckerberg bought 250,000+ MI300, and is buying MI355X for recommendation engines and Llama training. It is very unlikely for Daddy Zuck to slow down AMD Chips, due to its Inference superiority to NVDA Chips. Most likely we will see at least 300,000-400,000 MI355X ordered from now toward end of H1 2025. And another 300,000-500,000 MI450 by H2 2025. Or ~$20B from just Meta in H2 alone, excluded H1. 3. xAI : Musk confirmed "AMD GPUs work very well" for Grok's small/medium models, with 30% of Grok-1 on MI300X. xAI's Colossus (200K+ GPUs, targeting 1M) and Oracle partnership (via OCI's MI355X cluster) position it for MI450 trials in H1 2026. With $6B funding and Grok integration into Oracle services, xAI could allocate 10-20% ($10B-$15B) to MI450 for distributed inference. We haven't heard the detail from Daddy Elon Musk yet, but most likely not going to be spending less than OpenAI or Sam Altman 4. Oracle ($ORCL): A multi-billion-dollar MI355X deal powers OCI's AI superclusters, with $500B+ remaining performance obligations. Larry Ellison's zettascale ambitions and xAI/OpenAI integrations make Oracle a MI450 anchor tenant—projected 50-100k units ($15B+ spend) for enterprise AI platforms. $ORCL is likely to spend more on the new "secret weapon" due to its capability in AI inference and cost advantage for $500B backlog. 5. Others ( Microsoft , Amazon , Saudi+other countries): Microsoft (Azure MI300X for training) and Amazon ($148B 15-year spend) test MI450 via Stargate ($500B with Oracle/SoftBank). Emerging buyers like G42 (5GW UAE campus), Crusoe, and Hot Aisle add 5-10GW demand. These potentially would add $15B-$30B in 2026 alone. We also need to factor in $TSM supply constraint( $NVDA is TSMC favorite), so $AMD market cap/growth is being tamed by TSMC. So what are you saying Mike, well $AMD 2026 revenue could hit $90-$100B by end of 2026 or nearly 185% growth YoYo. So what does that mean for valuation? I have no idea how Mr. Market gonna value AMD in 2026 with 3 digits growth. My Conservative $620 was my best projection until today with OpenAI partnership. I'm telling you as one of the biggest AMD bull, that I will leave it to "smart money" and other investors to do the price discovery while I'm chilling and writing DDs daily. Lastly, AMD's MI450 isn't hype—it's a calibrated strike at Nvidia's vulnerabilities, amplified by hyperscaler bets like Meta's 42% allocation and OpenAI's 6GW lifeline. By prioritizing inference efficiency, rack-scale innovation, and open ecosystems, AMD will siphon 10-15% share in 2026, scaling to 20%+ as TCO trumps CUDA loyalty. Meta, xAI, Oracle et al. aren't passive; they're active co-designers, betting billions on MI450 to fuel AGI pursuits without Nvidia's premium. For investors, this is AMD's inflection Per Dr. Lisa Su Not Financial Advice!
1:06

$AMD $620/share is too conservative for 2026 🧵 Some quick facts before I dive into this super long thread: $META allocated 42% GPUs to $AMD and 58% to $NVDA OpenAI allocated 6GW(38%) to $AMD and 10GW to $NVDA My $620 PT below by end of 2026 was only for 10-15% market share. I believe $AMD is going to have much much higher market share than I projected. The AI accelerator market is exploding, projected to reach $500 billion by 2028(is now heading $1Tril), driven by insatiable demand for training and inference compute in large language models (LLMs), recommendation systems, and autonomous systems. Nvidia ($NVDA) has long held a stranglehold, commanding over 90% market share through its CUDA ecosystem and superior rack-scale solutions. However, AMD is mounting a formidable challenge, leveraging cost advantages, open-source software momentum, and hyperscaler partnerships to erode Nvidia's moat. Recent deals—such as Meta's ($META) allocation of 42% of its GPU capacity to AMD and OpenAI's commitment to 6GW of AMD compute (versus 10GW for Nvidia)—signal a tipping point. At the forefront is AMD's Instinct MI450 series, a next-generation AI GPU slated for H2 2026 launch, which promises "no-excuses" leadership in training, inference, and distributed workloads. This analysis dissects how AMD will capture more market share and why hyperscalers like $Meta , xAI , Oracle , and others are poised to become voracious buyers of the MI450. AMD's AI GPU revenue has surged from negligible levels in 2022 to an estimated $4-5 billion in 2025, capturing ~6% of the data center GPU market. This growth stems from the Instinct MI300X, which offers 141GB of HBM3 memory and competitive FP8/FP16 performance at 20-30% lower cost than Nvidia's H100. Hyperscalers, facing NVIDIA 's overcharging, have turned to AMD for diversification. Meta, for instance, plans 600,000 H100-equivalent GPUs by end-2024, with ~42% (or 250,000+ units) sourced from AMD's MI300 series for inference tasks like image editing and AI assistants. Similarly, OpenAI's recent multi-year deal commits to 6GW of AMD compute—equivalent to ~300,000-400,000 MI450 GPUs—starting with 1GW in 2026, explicitly to counterbalance its 10GW Nvidia allocation. These aren't one-offs. Microsoft Azure, Amazon AWS, and Oracle Cloud Infrastructure (OCI) have integrated MI300X for AI workloads, with Oracle deploying 30,000 MI355X units in zettascale clusters. xAI, Elon Musk Musk's AI venture, ran 30% of Grok-1's production traffic on MI300X GPUs and has confirmed ongoing purchases. Collectively, these partners represent over $400 billion in projected AI infrastructure spend through 2028, with AMD targeting up to 40% market share. For those that subscribed, I wrote a specific thread on how AMD "secret weapon" is going to change the game in 2026 with an improved designs on all its products, yes AMD has patent on it. Software is the linchpin. AMD's ROCm platform, once derided as "half-baked," now supports day-zero integration for Llama-4, DeepSeek V3, and GPT-OSS models—closing the CUDA gap. Benchmarks show MI355X (MI450 precursor) outperforming Nvidia's B200 in inference by 1.5-2x on memory-bound tasks, at 25-35% lower TCO. For training, MI450's rack-scale IF128 configuration (128 GPUs, 1.4 PB/s intra-rack bandwidth) rivals Nvidia's VR200 NVL144, enabling clusters like xAI's Colossus (scaling to 1M GPUs). My below thread projected Etimated conservative FY 25 revenue: $34-$36B Estimated conservative FY 26 revenue: $55B-$62B Below is why $AMD is revenue is going to be much higher after OpenAI deal. 1. OpenAI 1GW in 2026. With high demand for MI355X at $30,000k+ per unit, with MI450 is likely to be sold in the $45k-$55k. We can safely calcuate 1GW would require roughly 400,000 MI450 GPUs. or Roughly ~$20B revenue in 2026 alone from OpenAI. That would mean $AMD would hit $56B just from one partnership(OpenAI) in 2026 2. $META, the biggest spender on AI Infrastructure right now, Daddy Zuckerberg bought 250,000+ MI300, and is buying MI355X for recommendation engines and Llama training. It is very unlikely for Daddy Zuck to slow down AMD Chips, due to its Inference superiority to NVDA Chips. Most likely we will see at least 300,000-400,000 MI355X ordered from now toward end of H1 2025. And another 300,000-500,000 MI450 by H2 2025. Or ~$20B from just Meta in H2 alone, excluded H1. 3. xAI : Musk confirmed "AMD GPUs work very well" for Grok's small/medium models, with 30% of Grok-1 on MI300X. xAI's Colossus (200K+ GPUs, targeting 1M) and Oracle partnership (via OCI's MI355X cluster) position it for MI450 trials in H1 2026. With $6B funding and Grok integration into Oracle services, xAI could allocate 10-20% ($10B-$15B) to MI450 for distributed inference. We haven't heard the detail from Daddy Elon Musk yet, but most likely not going to be spending less than OpenAI or Sam Altman 4. Oracle ($ORCL): A multi-billion-dollar MI355X deal powers OCI's AI superclusters, with $500B+ remaining performance obligations. Larry Ellison's zettascale ambitions and xAI/OpenAI integrations make Oracle a MI450 anchor tenant—projected 50-100k units ($15B+ spend) for enterprise AI platforms. $ORCL is likely to spend more on the new "secret weapon" due to its capability in AI inference and cost advantage for $500B backlog. 5. Others ( Microsoft , Amazon , Saudi+other countries): Microsoft (Azure MI300X for training) and Amazon ($148B 15-year spend) test MI450 via Stargate ($500B with Oracle/SoftBank). Emerging buyers like G42 (5GW UAE campus), Crusoe, and Hot Aisle add 5-10GW demand. These potentially would add $15B-$30B in 2026 alone. We also need to factor in $TSM supply constraint( $NVDA is TSMC favorite), so $AMD market cap/growth is being tamed by TSMC. So what are you saying Mike, well $AMD 2026 revenue could hit $90-$100B by end of 2026 or nearly 185% growth YoYo. So what does that mean for valuation? I have no idea how Mr. Market gonna value AMD in 2026 with 3 digits growth. My Conservative $620 was my best projection until today with OpenAI partnership. I'm telling you as one of the biggest AMD bull, that I will leave it to "smart money" and other investors to do the price discovery while I'm chilling and writing DDs daily. Lastly, AMD's MI450 isn't hype—it's a calibrated strike at Nvidia's vulnerabilities, amplified by hyperscaler bets like Meta's 42% allocation and OpenAI's 6GW lifeline. By prioritizing inference efficiency, rack-scale innovation, and open ecosystems, AMD will siphon 10-15% share in 2026, scaling to 20%+ as TCO trumps CUDA loyalty. Meta, xAI, Oracle et al. aren't passive; they're active co-designers, betting billions on MI450 to fuel AGI pursuits without Nvidia's premium. For investors, this is AMD's inflection Per Dr. Lisa Su Not Financial Advice!

Mike

711,006 次观看 • 10 个月前

$AMD $5 Trillion is Inevitable LT| Agentic AI🧵 Agentic AI is the new $5 Trillion TAM 🚨🚨🚨 This thead will do Comp with $INTC and how to quantify this massive Agentic AI demand spike, and forcing Jensen to rush a CPU design. Global Agentic AI Market size is estimated to be $3-$5Trillion TAM by 2030(McKinsey) Quantifying the demand from agentic AI for AMD involves assessing the broader market growth for agentic systems, their unique computational requirements (particularly for CPUs in orchestration and reasoning tasks), and AMD's positioning very well through products like EPYC processors and partnerships. AMD EPYC Venice is the most superior choice in 2026-2027 for most Agentic AI workloads Agentic AI refers to autonomous AI agents that perform multi-step tasks, involving sequential logic, tool integration, and decision-making workloads that heavily rely on CPUs for handling orchestration, memory management, and context switching, rather than just GPU-parallelized training or batch inference. Agentic AI is often cited as 40-100x more "hungry" than traditional AI due to its continuous, 24/7 operation and complex workflows. This stems from factors like chain-of-thought reasoning (multiple LLM calls per query), API/tool interactions, memory management, and orchestration loops, which can generate 10-100x more tokens and require real-time responsiveness. For example, a single agentic query might trigger 5-20 model inferences, making it 10-20x more compute-intensive than simple chatbots, and the always-on nature compounds this to 40-100x overall. Nvidia's CEO has highlighted this as driving "easily 100x more computation" for inference in agentic/reasoning setups. AMD's EPYC Venice (6th Gen EPYC, codenamed "Venice") and Intel's Xeon 7 Diamond Rapids represent the pinnacle of server CPU technology in 2026, both targeting high-performance data center workloads like AI inference, agentic AI orchestration, cloud computing, and HPC. Venice builds on AMD's Zen 6 architecture, emphasizing core density and efficiency, while Diamond Rapids leverages Intel's Panther Cove P-cores for balanced performance. Both chips adopt similar advancements like 16-channel DDR5 memory and PCIe Gen 6, but differ in core counts, process nodes, and overall design philosophy. Intel has faced acute supply constraints across its Xeon lineup, including legacy nodes (Intel 7/3) and the ramping 18A process for next-gen parts. Intel shortage is expected with lead times up to 6 months or longer. 1. AMD EPYC Venice vs Intel Xeon 7 Diamond Rapids Architecture AMD: Zen 6 chiplet design with 8 CCDs and dual IODs Intel: Panther Cove P-cores; multi-die architecture with 4 compute tiles Core/Thread Count AMD: Up to 256 cores / 512 threads (Zen 6c variant) Intel: Up to 192 cores / 192 threads Process Node AMD: TSMC N2 (2nm) Intel: Intel 18A (1.8nm-class); in-house fab Memory Support AMD: 16-channel DDR5; up to 1.6 TB/s bandwidth. Intel: 16-channel DDR5 ; up to 1.6 TB/s bandwidth I/O and Connectivity AMD: PCIe Gen 6 (up to 128 lanes); twice the CPU-to-GPU bandwidth Intel: PCIe Gen 6 (up to 128 lanes); LGA 9324 socket Power (TDP) AMD: Starting 400-500W, potentially lower due to efficiency gains from TSMC 2nm Intel: Starting 400-500W, as it targets competitive efficiency Performance Projections AMD: Up to 70% uplift vs. 5th Gen Turin (1.7x in multi-threaded/AI tasks) Intel: ~40% faster than Granite Rapids (Xeon 6, 128-core). Lags AMD in per-core perf and 40-50% behind Venice core-for-core comp Target Workloads AMD: AI inference/orchestration, HPC, cloud virtualization. Partnerships Intel: Hyperscale AI, general enterprise. Custom silicon Pricing: AMD: estimated $10k-$20k for top SKUs Intel: estimated $8-$18k Availability: AMD: Significant Ramp H2 2026 due to higher allocation from TSMC Intel: H1-H2 2026 delayed, but trying to catch up Overall: ~Venice's 256 cores provide a 33% edge over Diamond Rapids' 192, making it superior for massively parallel tasks like AI training/inference or virtualization ~TSMC's N2 vs. Intel 18A debates rage on which is "better," but AMD's mature chiplet approach yields better density ( 32 cores/CCD vs. Intel's 48/tile). Venice's redesign reduces latency, aiding agentic AI where CPUs handle orchestration ~ Early projections show Venice widening AMD's lead matching or exceeding Diamond Rapids' perf with fewer watts in multi-threaded benchmarks. Intel's no-SMT design (to prioritize AI) handicaps it vs. AMD's 512 threads, though Clearwater Forest (E-core) could compete in density-focused niches. ~Power & Cooling: Both push above 400-500W, demanding liquid cooling. ~AMD been taking market share now above 40%. AMD EPYC Venice emerges as the superior choice in 2026 for most server workloads. Its higher core/thread count (256/512 vs. 192/192), stronger per-core performance, and architecture optimized for AI-driven tasks (agentic orchestration with GPU integration) provide decisive advantages in throughput, scalability, and efficiency. Projections indicate Venice delivering 1.7x the performance of prior gens while widening the gap over Intel ( 40-70% leads in multi-threaded benchmarks). AMD's fabless model with TSMC ensures reliable scaling, and its ecosystem ( open ROCm) appeals to AI adopters. Intel's Diamond Rapids is competitive in single-threaded enterprise apps and custom hyperscale ( NVLink), with potential fab advantages for supply/security. However, without SMT and lower density, it falls short in core-for-core battles—exposing Intel to another generation of AMD dominance unless 18A yields surprise efficiency gains. For data centers prioritizing raw compute ( AI, HPC), Venice wins; for Intel-centric ecosystems or specialized I/O, Diamond Rapids holds ground. Real benchmarks post-launch will confirm, but logic points to AMD pulling ahead. 2. Market size , Potential Revenue and Supply Global Agentic AI market size is projected to be $3-$5 Trillion by 2030 according to McKinsey, where consensus points to 40-50% CAGR driven by small to large enterprise demand. I also wrote a full thread on how and why Agentic AI is so explosive that AMD will blow all anlaysts estimate for subscribers. Link below if you are interested. AMD's data center segment hit a record $5.4B in Q4 2025 (up 39% YoY), with EPYC shipments ramping due to agentic demand. With 2GW of deployment in H2 2026, AMD AI data center revenue has $40-$50B+ at the lowest or most conservative projection; or Total Revenue in the $77-$94B For FY2026. However, Agentic AI massive demand spike could send EPYC revenue 3x to 4x in the next few years, potentially surpassing MI series GPU demand as enterprises prioritize CPU-dense Rack setups. This is pushing $NVDA Jensen to rush a CPU design and acquired Groq, a new CPU player due to this massive TAM. Noted that this is just popping just in weeks, highlighting we are just so early in this AI Supercycle and the pace of adoption is insane, and clearly productivity will skyrocket. Why? Because Agentic AI is 24/7 Smart AI agent working for you or your businesses is a mad compelling, and it is estimated to be 40-100x more Inference Hugnry! Many experts already said it is impossible to project this kind of Inference Demand. AI CapEx is expected to ramp up even more in 2027-2028-2029 and 2030 as Global Agentic AI is going to scale to $3-$5 Trillion TAM by 2030. The nature of Agentic is driving higher CPU/GPU ratio, with CPUs handling 50-90% of Agentic workflows. For example, The current Helios Rack: 18 compute trays per rack with 72 GPUs + 18 CPUs. The beauty of this $META and $AMD long term partnership is, that it is absolutely flexible to adjust racks to higher CPU rato or equal to service different needs. Helios rack can be easily swap to 2 GPUs 2CPUs or even CPUs only trays for dedicated orchestration/head nodes. You see, the beauty of this open rack-scale is flexibility and evolvability. If Agentic AI demand pushes much higher, AMD should be able to adjust variant trays without abandoning Heilos Rack. We can't talk just about massive Agentic AI demand without talking about the Supply side or TSMC. TSMC, AMD's primary foundry for advanced nodes ( Zen 6/Venice on N2/2nm), is addressing AI-driven shortages through massive expansions. TSMC accelerates fab construction with up to 10 facilities targeted for 2026. TSMC is accelerating its domestic manufacturing expansion, with industry sources indicating that as many as ten fabs could be under construction or preparing to begin operations across Taiwan’s major science parks. TSMC Capex: $52-56B in 2026 (up 37% YoY), with $45B already approved for new/upgraded capacities. 70-80% for advanced processes (2nm/A16), 10-20% for packaging (CoWoS quadrupling to 120-140K wafers/month by late 2026). In addition, Taiwanese companies (led by TSMC) commit to at least $250B in direct investments in US-based advanced semiconductor, AI, and energy production/innovation capacity.Taiwan provides $250B in government credit guarantees to facilitate additional investments and build a full US semiconductor ecosystem (including industrial parks). TSMC completed a second land purchase in Arizona (January 2026) for gigafab scaling, with an additional $100B+ (potentially four more modules) to further expand and qualify for tariff exemptions. AMD with secured 12GW from OpenAI and $META and massive Agentic AI will mean higher priority acess to 20-30% more wafers on TSMC advanced nodes, as TSMC has multi-year agreements with AMD for AI chips. Dr. C. C. Wei, CEO of TSMC quote: "I spend a lot of time in the last three or four months talking to my customer and then customers. Customer. I want to make sure that my customers demand are real. I talk to those cloud service providers, all of them. Their answer is. I'm quite satisfied with their answer. Actually they show me the evidence that the AI really help their business. So they grow their business successfully and he or she in their financial return. So I also double check their financial status. They are very rich." Amid shortages, the US buildout ensures AMD can ramp production of Instinct GPUs and EPYC CPUs without the constraints hitting competitors like Intel. By diversifying away from Taiwan (85% of advanced nodes today), the agreement mitigates supply disruptions, ensuring stable flows for AMD's chips. Scaling production and securing supply will matter for AMD the most in the next 5-10 years growth. The growth could be 80-100% YoY or higher; or it could be in the 60%. The aggressive TSMC supply ramp is reassuring the higher growth point. Conclusion: AMD stands at a pivotal inflection point in 2026, where the explosive rise of agentic AI demanding 40-100x more inference compute through its 24/7, multi-step orchestration positions the company to potentially triple its EPYC CPU revenue to $45-60B+ by 2028 while scaling Instinct GPUs to tens of billions annually by 2027. Agentic AI demand could push AI CapEx closer to $1 Trillion in 2027, far higher than most estimates. Dr. Lisa Su, AMD's visionary CEO, is masterfully securing supply to harness this massive demand by prioritizing operational execution and deep TSMC collaboration, ensuring readiness for the second-half 2026 AI ramp. Dr. Su has explicitly called out surging EPYC demand for agentic tasks where CPUs power head nodes and traditional workloads alongside GPUs while guiding for data center dominance through proactive capacity planning and partnerships like Nutanix ($150M investment for open agentic platforms) or providing tens of millions CPUs for OpenAI, $META, $ORCL, $AMZN, $MSFT, $GOOGL and others. Her strategy includes multi-year TSMC agreements for advanced nodes (N2 for Venice CPUs and future Instincts), diversifying beyond Taiwan to mitigate risks, and unveiling innovations like the MI455X GPU at CES 2026, which she touted as enabling "the next trillion-dollar market opportunity" in physical AI. Dr. Su's forward-looking vision predicting AI reaching 5 billion users emphasizes "AI everywhere," backed by hardware like Ryzen AI chips, all while declaring demand "going through the roof" and committing to scale without bottlenecks. TSMC's aggressive ramp-up, fueled by $52-56B in 2026 capex (up 37% YoY) and 10+ new fabs across Taiwan, the US (Arizona cluster expanding to 6+ modules with $165B+ investment), Japan, and Europe, provides profound reassurance for AMD's supply stability. The January 2026 US-Taiwan agreement committing $250B in investments and credit guarantees for US reshoring accelerates this, granting tariff relief (15% rates with 1.5-2.5x exemptions) tied to capacity buildouts, enabling TSMC to potentially double output over the decade to meet AI wafer hunger. This translates to 20-30% higher wafer allocations on key nodes, sidestepping Intel-like shortages and empowering Dr. Su's team to deliver on hyperscaler demands without disruption. Ultimately, this synergy cements AMD's leadership in the agentic era, promising sustained growth, $5T+ valuations at scale, and a resilient path forward as AI reshapes the world. This is NOT Financial Advice! Video source: AMD CES 2026
2:49

$AMD $5 Trillion is Inevitable LT| Agentic AI🧵 Agentic AI is the new $5 Trillion TAM 🚨🚨🚨 This thead will do Comp with $INTC and how to quantify this massive Agentic AI demand spike, and forcing Jensen to rush a CPU design. Global Agentic AI Market size is estimated to be $3-$5Trillion TAM by 2030(McKinsey) Quantifying the demand from agentic AI for AMD involves assessing the broader market growth for agentic systems, their unique computational requirements (particularly for CPUs in orchestration and reasoning tasks), and AMD's positioning very well through products like EPYC processors and partnerships. AMD EPYC Venice is the most superior choice in 2026-2027 for most Agentic AI workloads Agentic AI refers to autonomous AI agents that perform multi-step tasks, involving sequential logic, tool integration, and decision-making workloads that heavily rely on CPUs for handling orchestration, memory management, and context switching, rather than just GPU-parallelized training or batch inference. Agentic AI is often cited as 40-100x more "hungry" than traditional AI due to its continuous, 24/7 operation and complex workflows. This stems from factors like chain-of-thought reasoning (multiple LLM calls per query), API/tool interactions, memory management, and orchestration loops, which can generate 10-100x more tokens and require real-time responsiveness. For example, a single agentic query might trigger 5-20 model inferences, making it 10-20x more compute-intensive than simple chatbots, and the always-on nature compounds this to 40-100x overall. Nvidia's CEO has highlighted this as driving "easily 100x more computation" for inference in agentic/reasoning setups. AMD's EPYC Venice (6th Gen EPYC, codenamed "Venice") and Intel's Xeon 7 Diamond Rapids represent the pinnacle of server CPU technology in 2026, both targeting high-performance data center workloads like AI inference, agentic AI orchestration, cloud computing, and HPC. Venice builds on AMD's Zen 6 architecture, emphasizing core density and efficiency, while Diamond Rapids leverages Intel's Panther Cove P-cores for balanced performance. Both chips adopt similar advancements like 16-channel DDR5 memory and PCIe Gen 6, but differ in core counts, process nodes, and overall design philosophy. Intel has faced acute supply constraints across its Xeon lineup, including legacy nodes (Intel 7/3) and the ramping 18A process for next-gen parts. Intel shortage is expected with lead times up to 6 months or longer. 1. AMD EPYC Venice vs Intel Xeon 7 Diamond Rapids Architecture AMD: Zen 6 chiplet design with 8 CCDs and dual IODs Intel: Panther Cove P-cores; multi-die architecture with 4 compute tiles Core/Thread Count AMD: Up to 256 cores / 512 threads (Zen 6c variant) Intel: Up to 192 cores / 192 threads Process Node AMD: TSMC N2 (2nm) Intel: Intel 18A (1.8nm-class); in-house fab Memory Support AMD: 16-channel DDR5; up to 1.6 TB/s bandwidth. Intel: 16-channel DDR5 ; up to 1.6 TB/s bandwidth I/O and Connectivity AMD: PCIe Gen 6 (up to 128 lanes); twice the CPU-to-GPU bandwidth Intel: PCIe Gen 6 (up to 128 lanes); LGA 9324 socket Power (TDP) AMD: Starting 400-500W, potentially lower due to efficiency gains from TSMC 2nm Intel: Starting 400-500W, as it targets competitive efficiency Performance Projections AMD: Up to 70% uplift vs. 5th Gen Turin (1.7x in multi-threaded/AI tasks) Intel: ~40% faster than Granite Rapids (Xeon 6, 128-core). Lags AMD in per-core perf and 40-50% behind Venice core-for-core comp Target Workloads AMD: AI inference/orchestration, HPC, cloud virtualization. Partnerships Intel: Hyperscale AI, general enterprise. Custom silicon Pricing: AMD: estimated $10k-$20k for top SKUs Intel: estimated $8-$18k Availability: AMD: Significant Ramp H2 2026 due to higher allocation from TSMC Intel: H1-H2 2026 delayed, but trying to catch up Overall: ~Venice's 256 cores provide a 33% edge over Diamond Rapids' 192, making it superior for massively parallel tasks like AI training/inference or virtualization ~TSMC's N2 vs. Intel 18A debates rage on which is "better," but AMD's mature chiplet approach yields better density ( 32 cores/CCD vs. Intel's 48/tile). Venice's redesign reduces latency, aiding agentic AI where CPUs handle orchestration ~ Early projections show Venice widening AMD's lead matching or exceeding Diamond Rapids' perf with fewer watts in multi-threaded benchmarks. Intel's no-SMT design (to prioritize AI) handicaps it vs. AMD's 512 threads, though Clearwater Forest (E-core) could compete in density-focused niches. ~Power & Cooling: Both push above 400-500W, demanding liquid cooling. ~AMD been taking market share now above 40%. AMD EPYC Venice emerges as the superior choice in 2026 for most server workloads. Its higher core/thread count (256/512 vs. 192/192), stronger per-core performance, and architecture optimized for AI-driven tasks (agentic orchestration with GPU integration) provide decisive advantages in throughput, scalability, and efficiency. Projections indicate Venice delivering 1.7x the performance of prior gens while widening the gap over Intel ( 40-70% leads in multi-threaded benchmarks). AMD's fabless model with TSMC ensures reliable scaling, and its ecosystem ( open ROCm) appeals to AI adopters. Intel's Diamond Rapids is competitive in single-threaded enterprise apps and custom hyperscale ( NVLink), with potential fab advantages for supply/security. However, without SMT and lower density, it falls short in core-for-core battles—exposing Intel to another generation of AMD dominance unless 18A yields surprise efficiency gains. For data centers prioritizing raw compute ( AI, HPC), Venice wins; for Intel-centric ecosystems or specialized I/O, Diamond Rapids holds ground. Real benchmarks post-launch will confirm, but logic points to AMD pulling ahead. 2. Market size , Potential Revenue and Supply Global Agentic AI market size is projected to be $3-$5 Trillion by 2030 according to McKinsey, where consensus points to 40-50% CAGR driven by small to large enterprise demand. I also wrote a full thread on how and why Agentic AI is so explosive that AMD will blow all anlaysts estimate for subscribers. Link below if you are interested. AMD's data center segment hit a record $5.4B in Q4 2025 (up 39% YoY), with EPYC shipments ramping due to agentic demand. With 2GW of deployment in H2 2026, AMD AI data center revenue has $40-$50B+ at the lowest or most conservative projection; or Total Revenue in the $77-$94B For FY2026. However, Agentic AI massive demand spike could send EPYC revenue 3x to 4x in the next few years, potentially surpassing MI series GPU demand as enterprises prioritize CPU-dense Rack setups. This is pushing $NVDA Jensen to rush a CPU design and acquired Groq, a new CPU player due to this massive TAM. Noted that this is just popping just in weeks, highlighting we are just so early in this AI Supercycle and the pace of adoption is insane, and clearly productivity will skyrocket. Why? Because Agentic AI is 24/7 Smart AI agent working for you or your businesses is a mad compelling, and it is estimated to be 40-100x more Inference Hugnry! Many experts already said it is impossible to project this kind of Inference Demand. AI CapEx is expected to ramp up even more in 2027-2028-2029 and 2030 as Global Agentic AI is going to scale to $3-$5 Trillion TAM by 2030. The nature of Agentic is driving higher CPU/GPU ratio, with CPUs handling 50-90% of Agentic workflows. For example, The current Helios Rack: 18 compute trays per rack with 72 GPUs + 18 CPUs. The beauty of this $META and $AMD long term partnership is, that it is absolutely flexible to adjust racks to higher CPU rato or equal to service different needs. Helios rack can be easily swap to 2 GPUs 2CPUs or even CPUs only trays for dedicated orchestration/head nodes. You see, the beauty of this open rack-scale is flexibility and evolvability. If Agentic AI demand pushes much higher, AMD should be able to adjust variant trays without abandoning Heilos Rack. We can't talk just about massive Agentic AI demand without talking about the Supply side or TSMC. TSMC, AMD's primary foundry for advanced nodes ( Zen 6/Venice on N2/2nm), is addressing AI-driven shortages through massive expansions. TSMC accelerates fab construction with up to 10 facilities targeted for 2026. TSMC is accelerating its domestic manufacturing expansion, with industry sources indicating that as many as ten fabs could be under construction or preparing to begin operations across Taiwan’s major science parks. TSMC Capex: $52-56B in 2026 (up 37% YoY), with $45B already approved for new/upgraded capacities. 70-80% for advanced processes (2nm/A16), 10-20% for packaging (CoWoS quadrupling to 120-140K wafers/month by late 2026). In addition, Taiwanese companies (led by TSMC) commit to at least $250B in direct investments in US-based advanced semiconductor, AI, and energy production/innovation capacity.Taiwan provides $250B in government credit guarantees to facilitate additional investments and build a full US semiconductor ecosystem (including industrial parks). TSMC completed a second land purchase in Arizona (January 2026) for gigafab scaling, with an additional $100B+ (potentially four more modules) to further expand and qualify for tariff exemptions. AMD with secured 12GW from OpenAI and $META and massive Agentic AI will mean higher priority acess to 20-30% more wafers on TSMC advanced nodes, as TSMC has multi-year agreements with AMD for AI chips. Dr. C. C. Wei, CEO of TSMC quote: "I spend a lot of time in the last three or four months talking to my customer and then customers. Customer. I want to make sure that my customers demand are real. I talk to those cloud service providers, all of them. Their answer is. I'm quite satisfied with their answer. Actually they show me the evidence that the AI really help their business. So they grow their business successfully and he or she in their financial return. So I also double check their financial status. They are very rich." Amid shortages, the US buildout ensures AMD can ramp production of Instinct GPUs and EPYC CPUs without the constraints hitting competitors like Intel. By diversifying away from Taiwan (85% of advanced nodes today), the agreement mitigates supply disruptions, ensuring stable flows for AMD's chips. Scaling production and securing supply will matter for AMD the most in the next 5-10 years growth. The growth could be 80-100% YoY or higher; or it could be in the 60%. The aggressive TSMC supply ramp is reassuring the higher growth point. Conclusion: AMD stands at a pivotal inflection point in 2026, where the explosive rise of agentic AI demanding 40-100x more inference compute through its 24/7, multi-step orchestration positions the company to potentially triple its EPYC CPU revenue to $45-60B+ by 2028 while scaling Instinct GPUs to tens of billions annually by 2027. Agentic AI demand could push AI CapEx closer to $1 Trillion in 2027, far higher than most estimates. Dr. Lisa Su, AMD's visionary CEO, is masterfully securing supply to harness this massive demand by prioritizing operational execution and deep TSMC collaboration, ensuring readiness for the second-half 2026 AI ramp. Dr. Su has explicitly called out surging EPYC demand for agentic tasks where CPUs power head nodes and traditional workloads alongside GPUs while guiding for data center dominance through proactive capacity planning and partnerships like Nutanix ($150M investment for open agentic platforms) or providing tens of millions CPUs for OpenAI, $META, $ORCL, $AMZN, $MSFT, $GOOGL and others. Her strategy includes multi-year TSMC agreements for advanced nodes (N2 for Venice CPUs and future Instincts), diversifying beyond Taiwan to mitigate risks, and unveiling innovations like the MI455X GPU at CES 2026, which she touted as enabling "the next trillion-dollar market opportunity" in physical AI. Dr. Su's forward-looking vision predicting AI reaching 5 billion users emphasizes "AI everywhere," backed by hardware like Ryzen AI chips, all while declaring demand "going through the roof" and committing to scale without bottlenecks. TSMC's aggressive ramp-up, fueled by $52-56B in 2026 capex (up 37% YoY) and 10+ new fabs across Taiwan, the US (Arizona cluster expanding to 6+ modules with $165B+ investment), Japan, and Europe, provides profound reassurance for AMD's supply stability. The January 2026 US-Taiwan agreement committing $250B in investments and credit guarantees for US reshoring accelerates this, granting tariff relief (15% rates with 1.5-2.5x exemptions) tied to capacity buildouts, enabling TSMC to potentially double output over the decade to meet AI wafer hunger. This translates to 20-30% higher wafer allocations on key nodes, sidestepping Intel-like shortages and empowering Dr. Su's team to deliver on hyperscaler demands without disruption. Ultimately, this synergy cements AMD's leadership in the agentic era, promising sustained growth, $5T+ valuations at scale, and a resilient path forward as AI reshapes the world. This is NOT Financial Advice! Video source: AMD CES 2026

Mike

44,460 次观看 • 5 个月前

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!
3:09

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!

Mike

102,223 次观看 • 7 个月前

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.
0:23

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.

Varun

37,555 次观看 • 5 个月前