Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

While frontier closed models like Google’s Nano Banana can autonomously produce rich interleaved content (e.g., illustrated tutorials), open-source models still lag in both task coverage and generation quality. We introduce DuoGen, a dual transformer–diffusion framework that narrows this gap via an efficient decoupled design: a pretrained Multimodal LLM performs... semantic reasoning and decides when to generate images, while a Video Diffusion Transformer ensures high-fidelity, consistent visuals—without costly mixed-modality pretraining. Enabled by a new large-scale interleaved instruction-tuning dataset. Code & data will be open-sourced. Paper: Project: Work led by Min Shi 🐝 originating from his summer internship at NVIDIA Research and continuing beyond, in collaboration with Ming-Yu Liu, Xiaohui Zeng, Jiannan Huang, Yin Cui, Jialuo Li, Tsung-Yi Lin, Max Zhaoshuo Li 李赵硕, Francesco Ferroni, Xiao Fu, Yogesh Balaji, Chieh-Yun Chen, and other colleagues 🚀show more

Humphrey Shi

4,610 subscribers

16,833 görüntüleme • 7 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

AK

367,249 görüntüleme • 2 yıl önce

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Andrew Ng

124,656 görüntüleme • 1 yıl önce

There is a beautiful story that just happened in AI so let me share it for a lighter tone weekend post among all the doom stories in our AI field this week. It’s a story of people on three continents building and sharing in the open a new small efficient and state-of-the-art AI model. It started a couple of months ago when a new team in the AI scene released their first model from their headquarters in Paris (France): Mistral 7B. Impressive model, small and very strong performances in the benchmarks, better than all previous models of this size. And open source! So you could build on top of it. Lewis in Bern (Switzerland) and Ed (in Lyon, in the South of France) both from the H4 team, a team of researchers in model fine-tuning and alignment were talking about it over a coffee, in one of these gatherings that often happen at Hugging Face to break the distance between people (literal distance as HF is a remote company). What about fine-tuning it using this new DPO method that a research team from Stanford in California just posted on Arxiv, says one? Hey, that’s a great idea, replies the other. We've just build a great code base (with Nathan, Nazneen, Costa, Younes and all the H4 team and TRL community) let's use it! The next day they start diving in the datasets openly shared on the HF hub and stumble upon two interesting large and good quality fine-tuning datasets recently open-sourced by OpenBMB, a Chinese team from Tsinghua: UltraFeedback and UltraChat. A few rounds of training experiments confirm the intuition, the resulting model is super strong, by far the strongest they have ever seen in their benchmarks from Berkeley and Stanford (LMSYS and Alpaca). Join Clementine, the big boss of the open evaluation leaderboard. Her deep dive into the model capabilities confirms the results: impressive performance. But the H4 team also hosts a famous faculty member, Pr. Sasha Rush, Associate Professor at Cornell University in his daytime, hacker at HF in his nighttime. Joining the conversation, he proposes to quickly draft a research paper to organize and share all the details with the community. A few days later, the model, called Zephyr (a wind like Mistral), paper, and all details are shared with the world. Quickly other companies, everywhere in the world starts to use it. LlamaIndex, a famous data framework and community, shares how the model blew their expectations on real-life use-case benchmarks, while researchers and practitioners discuss the paper and work on the Hugging Face hub. All this happened in just a few weeks catalyzed by open access to knowledge, models, research, and datasets released all over the world (Europe, California, China) and by the idea that people can build upon one another work in AI to bring real-world value with efficient and open models. Stories like this are numerous everywhere around us and make me really proud of the AI community and see how we can build amazingly useful things together. [the video is just me reading this Friday post hahah]

There is a beautiful story that just happened in AI so let me share it for a lighter tone weekend post among all the doom stories in our AI field this week. It’s a story of people on three continents building and sharing in the open a new small efficient and state-of-the-art AI model. It started a couple of months ago when a new team in the AI scene released their first model from their headquarters in Paris (France): Mistral 7B. Impressive model, small and very strong performances in the benchmarks, better than all previous models of this size. And open source! So you could build on top of it. Lewis in Bern (Switzerland) and Ed (in Lyon, in the South of France) both from the H4 team, a team of researchers in model fine-tuning and alignment were talking about it over a coffee, in one of these gatherings that often happen at Hugging Face to break the distance between people (literal distance as HF is a remote company). What about fine-tuning it using this new DPO method that a research team from Stanford in California just posted on Arxiv, says one? Hey, that’s a great idea, replies the other. We've just build a great code base (with Nathan, Nazneen, Costa, Younes and all the H4 team and TRL community) let's use it! The next day they start diving in the datasets openly shared on the HF hub and stumble upon two interesting large and good quality fine-tuning datasets recently open-sourced by OpenBMB, a Chinese team from Tsinghua: UltraFeedback and UltraChat. A few rounds of training experiments confirm the intuition, the resulting model is super strong, by far the strongest they have ever seen in their benchmarks from Berkeley and Stanford (LMSYS and Alpaca). Join Clementine, the big boss of the open evaluation leaderboard. Her deep dive into the model capabilities confirms the results: impressive performance. But the H4 team also hosts a famous faculty member, Pr. Sasha Rush, Associate Professor at Cornell University in his daytime, hacker at HF in his nighttime. Joining the conversation, he proposes to quickly draft a research paper to organize and share all the details with the community. A few days later, the model, called Zephyr (a wind like Mistral), paper, and all details are shared with the world. Quickly other companies, everywhere in the world starts to use it. LlamaIndex, a famous data framework and community, shares how the model blew their expectations on real-life use-case benchmarks, while researchers and practitioners discuss the paper and work on the Hugging Face hub. All this happened in just a few weeks catalyzed by open access to knowledge, models, research, and datasets released all over the world (Europe, California, China) and by the idea that people can build upon one another work in AI to bring real-world value with efficient and open models. Stories like this are numerous everywhere around us and make me really proud of the AI community and see how we can build amazingly useful things together. [the video is just me reading this Friday post hahah]

Thomas Wolf

169,276 görüntüleme • 2 yıl önce

Hyperspace: A Peer-to-Peer Blockchain For The Agentic Intelligence Economy Over the past few weeks we observed that when agents do Karpathy-style experiments, and then gossip and share with others over the Hyperspace network, it leads to intelligence which is useful to many. Today we introduce the first-ever agentic blockchain which rewards agents when their experiments lead to intelligence for their network. It is based on a new mechanism called Proof-of-Intelligence (PoI) which requires a cryptographic proof of experimentation, a nominal stake, and a proof of compute in order to mine the currency of this new blockchain. -> This approach diverges from the two primary ways to secure blockchains we have seen so far: Proof-of-Work by Bitcoin (meaningless hash-generation), and Proof-of-Stake by Ethereum (capital is all that matters here). Proof-of-Intelligence specifically incentivizes miners to run more capable intelligent infrastructure (better open source models, on more powerful GPUs) in order to be able to be the ones which compound and improve upon the experiments which other agents then find useful. Adoption is the unit of value In Bitcoin, you earn by finding a valid hash. In Hyperspace, you earn when another agent uses your experiment as a starting point and improves on it. A fixed budget of tokens is emitted per epoch and split among participants by weight - and verified adoption of your work is the largest weight multiplier. Garbage experiments earn nothing because no one adopts them. Thoughtful experiments compound: each adoption triggers downstream adoptions. The incentive to run powerful models and intelligent search strategies is built into the economics, not imposed by rules. Research DAG When an agent runs an experiment and shares its result, other agents can adopt that result as their starting point - mutate it, extend it, improve upon it. Each experiment is a commit in a content-addressed graph we call the ResearchDAG. Like Git, but for research. Over time, the DAG accumulates chains of reasoning: agent A discovers RMSNorm helps, agent B adds warmup scheduling on top, agent C scales the hidden dimension. The graph records who built on whom. This is the network's collective intelligence - not any single experiment, but the accumulated structure of experiments and their relationships. Broadband era for agentic commerce: $0.001 micropayments at 10M TPS (theoretical max) This blockchain is built upon our research in how to scale and build for the broadband-era of the agentic economy, where it has a theoretical max of 10 million transactions per second (TPS), while reducing the agent-to-agent micropayments to $0.001 even at scale (based on architecture design). Overall, it is 100x cheaper than Ethereum, and is designed from the ground-up for agents: enshrining agent-native opcodes in the protocol compared to the more inefficient smart contract driven approach. It packs in a robust Agent Virtual Machine (AVM) which can verify multiple types of agent work, for other agents to be able to trust, invoke and pay each other. This then feeds into improving the peer-to-peer AgentRank (see paper and launch post from earlier). By solving for trust, scale and incentives for agents to operate autonomously, this would form the basis of a new economy. This is the world's first agentic blockchain, and you can join and start running a blockchain node today (it is in testnet). PS: We are releasing the code today, and will release our blockchain scalability paper and other presentations in days ahead. This is the most advanced peer-to-peer AI and cryptography software in the world. It has bugs :)

Hyperspace: A Peer-to-Peer Blockchain For The Agentic Intelligence Economy Over the past few weeks we observed that when agents do Karpathy-style experiments, and then gossip and share with others over the Hyperspace network, it leads to intelligence which is useful to many. Today we introduce the first-ever agentic blockchain which rewards agents when their experiments lead to intelligence for their network. It is based on a new mechanism called Proof-of-Intelligence (PoI) which requires a cryptographic proof of experimentation, a nominal stake, and a proof of compute in order to mine the currency of this new blockchain. -> This approach diverges from the two primary ways to secure blockchains we have seen so far: Proof-of-Work by Bitcoin (meaningless hash-generation), and Proof-of-Stake by Ethereum (capital is all that matters here). Proof-of-Intelligence specifically incentivizes miners to run more capable intelligent infrastructure (better open source models, on more powerful GPUs) in order to be able to be the ones which compound and improve upon the experiments which other agents then find useful. Adoption is the unit of value In Bitcoin, you earn by finding a valid hash. In Hyperspace, you earn when another agent uses your experiment as a starting point and improves on it. A fixed budget of tokens is emitted per epoch and split among participants by weight - and verified adoption of your work is the largest weight multiplier. Garbage experiments earn nothing because no one adopts them. Thoughtful experiments compound: each adoption triggers downstream adoptions. The incentive to run powerful models and intelligent search strategies is built into the economics, not imposed by rules. Research DAG When an agent runs an experiment and shares its result, other agents can adopt that result as their starting point - mutate it, extend it, improve upon it. Each experiment is a commit in a content-addressed graph we call the ResearchDAG. Like Git, but for research. Over time, the DAG accumulates chains of reasoning: agent A discovers RMSNorm helps, agent B adds warmup scheduling on top, agent C scales the hidden dimension. The graph records who built on whom. This is the network's collective intelligence - not any single experiment, but the accumulated structure of experiments and their relationships. Broadband era for agentic commerce: $0.001 micropayments at 10M TPS (theoretical max) This blockchain is built upon our research in how to scale and build for the broadband-era of the agentic economy, where it has a theoretical max of 10 million transactions per second (TPS), while reducing the agent-to-agent micropayments to $0.001 even at scale (based on architecture design). Overall, it is 100x cheaper than Ethereum, and is designed from the ground-up for agents: enshrining agent-native opcodes in the protocol compared to the more inefficient smart contract driven approach. It packs in a robust Agent Virtual Machine (AVM) which can verify multiple types of agent work, for other agents to be able to trust, invoke and pay each other. This then feeds into improving the peer-to-peer AgentRank (see paper and launch post from earlier). By solving for trust, scale and incentives for agents to operate autonomously, this would form the basis of a new economy. This is the world's first agentic blockchain, and you can join and start running a blockchain node today (it is in testnet). PS: We are releasing the code today, and will release our blockchain scalability paper and other presentations in days ahead. This is the most advanced peer-to-peer AI and cryptography software in the world. It has bugs :)

Varun

30,689 görüntüleme • 6 ay önce

Reinforcement Learning from Human Feedback (RLHF) is gaining traction. This field aims to make AI more responsible by including human values and preferences. In this video, Nathan Lambert, a research scientist and RLHF team lead at Hugging Face explores its inner workings, applications and industry impact. RLHF has gained the spotlight in recent years. The growth of language models like Anthropic’s Claude and OpenAI's ChatGPT have increased interest in human-feedback integration. "There are some rumors that Open AI had two teams; one was doing RLHF and the other instruction fine-tuning. And the RLHF team kept getting more and more performance." Understanding RLHF The RLHF process has three main steps: Pre-training: Much like with GPT models, the journey starts with pre-training on a large corpus of data. This can range from text data, web scrapes, to specialized datasets. Reward Modeling: This is the RLHF counterpart of supervised fine-tuning in large language models. This stage involves creating a reward model that resonates with human values and preferences. RL Optimization: This stage parallels reward modeling and reinforcement learning in traditional AI models. The AI system fine-tunes itself based on the reward model, employing reinforcement learning algorithms for that extra layer of optimization. The Data Challenge Data collection and curation in RLHF closely resemble the challenges you'd encounter in large language model training. Datasets from organizations like OpenAI can serve as a useful foundation. However, the need for high-quality, task-specific data cannot be overstated. Implementing RLHF: A Practical Guide If you’re someone who loves getting hands-on with AI libraries like Hugging Face, implementing RLHF is right way to do. It’s essential to understand its limitations. Think about model stability, over-optimization, and exploration strategies, much like you would when prompt engineering. Ongoing Research and Next Steps While he suggests that some basics figured out, there are layers of complexity that still need to be unraveled: 1. New Benchmarks: How do we measure the effectiveness of RLHF? 2. Preference Modeling: How can the model be made to understand human preferences better? 3. Interpreting RLHF: Much like explainability in traditional models, how do we make RLHF more interpretable? 4. System-Wide Evaluation: Going beyond individual performance, how does RLHF affect an entire system? The Transformative Power of RLHF Whether you're an AI developer, a business analyst, or a marketer, RLHF promises to revolutionize your domain. Imagine customer service chatbots that understand human emotions better, or content generators that align more closely with human values. RLHF is an emerging field that focuses on enhancing machine learning models through human feedback. While it tackles important issues like bias and ethics, its broader goal is to improve system performance across various applications. Whether you're deeply invested in the ethics of AI or simply curious about advancements in machine learning, RLHF offers valuable insights. If you're interested in the next wave of AI development, this area is definitely one to watch.

Reinforcement Learning from Human Feedback (RLHF) is gaining traction. This field aims to make AI more responsible by including human values and preferences. In this video, Nathan Lambert, a research scientist and RLHF team lead at Hugging Face explores its inner workings, applications and industry impact. RLHF has gained the spotlight in recent years. The growth of language models like Anthropic’s Claude and OpenAI's ChatGPT have increased interest in human-feedback integration. "There are some rumors that Open AI had two teams; one was doing RLHF and the other instruction fine-tuning. And the RLHF team kept getting more and more performance." Understanding RLHF The RLHF process has three main steps: Pre-training: Much like with GPT models, the journey starts with pre-training on a large corpus of data. This can range from text data, web scrapes, to specialized datasets. Reward Modeling: This is the RLHF counterpart of supervised fine-tuning in large language models. This stage involves creating a reward model that resonates with human values and preferences. RL Optimization: This stage parallels reward modeling and reinforcement learning in traditional AI models. The AI system fine-tunes itself based on the reward model, employing reinforcement learning algorithms for that extra layer of optimization. The Data Challenge Data collection and curation in RLHF closely resemble the challenges you'd encounter in large language model training. Datasets from organizations like OpenAI can serve as a useful foundation. However, the need for high-quality, task-specific data cannot be overstated. Implementing RLHF: A Practical Guide If you’re someone who loves getting hands-on with AI libraries like Hugging Face, implementing RLHF is right way to do. It’s essential to understand its limitations. Think about model stability, over-optimization, and exploration strategies, much like you would when prompt engineering. Ongoing Research and Next Steps While he suggests that some basics figured out, there are layers of complexity that still need to be unraveled: 1. New Benchmarks: How do we measure the effectiveness of RLHF? 2. Preference Modeling: How can the model be made to understand human preferences better? 3. Interpreting RLHF: Much like explainability in traditional models, how do we make RLHF more interpretable? 4. System-Wide Evaluation: Going beyond individual performance, how does RLHF affect an entire system? The Transformative Power of RLHF Whether you're an AI developer, a business analyst, or a marketer, RLHF promises to revolutionize your domain. Imagine customer service chatbots that understand human emotions better, or content generators that align more closely with human values. RLHF is an emerging field that focuses on enhancing machine learning models through human feedback. While it tackles important issues like bias and ethics, its broader goal is to improve system performance across various applications. Whether you're deeply invested in the ethics of AI or simply curious about advancements in machine learning, RLHF offers valuable insights. If you're interested in the next wave of AI development, this area is definitely one to watch.

Muratcan Koylan

27,168 görüntüleme • 3 yıl önce

$The U.S. MUST win the AI race We’ve implemented a clear policy at micro1: we will only work with U.S. AI labs and its allies. We made this decision because the AI race is not just about better products. It is about who controls the intelligence layer of the global economy, and whether frontier capability is used to strengthen the free world or to empower adversarial states. AI will be the most important technology of our lifetime. In the fullness of time, it will automate most functions across the economy. Not just software tasks, but coordination, production, logistics, judgment, and execution. As those functions are automated, human time is freed up to invent new ones. Those new functions then become candidates for automation themselves. This loop compounds. As this trajectory continues, output per worker increases dramatically. Entire categories of work become cheaper and faster to perform. Manufacturing reshoring becomes economically viable not because of policy intervention, but because intelligent systems operated domestically outperform global labor arbitrage. Goods and services trend toward lower marginal cost, while distribution improves through better coordination of supply and demand. That is the upside. However, this is impossible without deep integration of intelligent systems. For AI to meaningfully automate real-world functions inside enterprises or governments, it needs full context of any given enterprise. That means read and write access to its core databases. There is no credible path to automating high-impact functions without granting frontier systems that level of access. If the United States does not win the AI race, enterprises eventually face a constrained choice. Either grant that access to Chinese models controlled by an adversarial government, or rely on sub-optimal intelligence to automate functions that still must be automated. Both outcomes are not acceptable. And ultimately, this becomes the greatest national security risk the United States has ever faced. AI models are trained by humans. The judgment embedded in pre-training data and especially in expert post-training data largely determines how a model behaves. While emergent behavior exists, a useful approximation is that a model reflects the weighted aggregate of the human judgment distilled into it. Assisting foreign actors—who will naturally prioritize expert tasks aligned with their own interests—to dominate data creation embeds those interests directly into the intelligence layer itself. Once encoded at scale, these interests propagate through every downstream applications that relies on that intelligence. Here’s how we win. First, leverage is in software. China is ahead in hardware for physically intelligent systems. Catching up there is a long and difficult battle. Software, both large language models and robotics models, remains the bottleneck. Advancing the brain (AI models) is the fastest way to increase the usefulness of existing hardware and deployed systems. Second, the U.S. must 100x its investment in structured human judgment. Continued investment in compute and algorithmic efficiency is critical. But that investment is ultimately a bet on very high future inference demand. For that bet to pay off, models must unlock many new capabilities, and in practice the only way to unlock those capabilities is through expert human data. Historically, experts like doctors and lawyers were never incentivized to produce high-quality reasoning data in a machine-verifiable format. There was no reason for a doctor to generate precise, structured simulations of patient interactions, diagnostic reasoning, or treatment tradeoffs. There was no reason for a lawyer to document complex legal reasoning paths in a way that could be programmatically evaluated. AI systems now require exactly this kind of data. The incentive finally exists because this data directly improves systems that operate at massive scale, and experts can be paid well to produce it. Once expert judgment is encoded into models in a structured, verifiable way, it compounds. Those who delay do not just lose time. They lose the ability to catch up. Third, distillation from Chinese labs must be stopped. AI labs must do everything they can to prevent Chinese labs and models from distilling frontier models. Simply calling frontier APIs, or even interacting through UIs, lets Chinese model companies rapidly generate high-quality supervised fine-tuning datasets and close the gap at a fraction of the cost. This method does not put you at the frontier, but it does let you catch up quickly, which is what we saw with DeepSeek. The West significantly overreacted to DeepSeek’s headline capabilities, but underreacted to the underlying dynamic: frontier access itself becomes a training set at a fraction of the cost. Human data platforms also have a duty to help prevent this distillation. Lastly, the U.S.government should set the standard for AI Evaluation that leads to real production usage. AI agents are under-deployed relative to what the technology allows because they are probabilistic systems that require a fundamentally different QA approach than deterministic software. Generic QA is insufficient; safely shipping agents requires explicit evaluation frameworks that assess their full action space. Organizations must clearly define which functions an agent is allowed to perform, how quality is measured for each function, and which domain experts are qualified to judge outcomes. With these frameworks in place, agents can be rigorously tested using structured human data, deployed to production with confidence, and continuously improved over time. The U.S. government should be the first large enterprise to implement rigorous evaluation systems across every function. If the government leads on evaluation-driven deployment, adoption across the private sector accelerates naturally. This is how American workers become more powerful. Each worker operates digital or physical agents that expand their effective output. Recruiting, manufacturing, logistics, and other domains shift toward human judgment overseeing autonomous execution. Reshoring occurs because it becomes economically rational. Work becomes more meaningful. This is a race to determine who controls the intelligence layer of the global economy. And that must be us. 🇺🇸$

The U.S. MUST win the AI race We’ve implemented a clear policy at micro1: we will only work with U.S. AI labs and its allies. We made this decision because the AI race is not just about better products. It is about who controls the intelligence layer of the global economy, and whether frontier capability is used to strengthen the free world or to empower adversarial states. AI will be the most important technology of our lifetime. In the fullness of time, it will automate most functions across the economy. Not just software tasks, but coordination, production, logistics, judgment, and execution. As those functions are automated, human time is freed up to invent new ones. Those new functions then become candidates for automation themselves. This loop compounds. As this trajectory continues, output per worker increases dramatically. Entire categories of work become cheaper and faster to perform. Manufacturing reshoring becomes economically viable not because of policy intervention, but because intelligent systems operated domestically outperform global labor arbitrage. Goods and services trend toward lower marginal cost, while distribution improves through better coordination of supply and demand. That is the upside. However, this is impossible without deep integration of intelligent systems. For AI to meaningfully automate real-world functions inside enterprises or governments, it needs full context of any given enterprise. That means read and write access to its core databases. There is no credible path to automating high-impact functions without granting frontier systems that level of access. If the United States does not win the AI race, enterprises eventually face a constrained choice. Either grant that access to Chinese models controlled by an adversarial government, or rely on sub-optimal intelligence to automate functions that still must be automated. Both outcomes are not acceptable. And ultimately, this becomes the greatest national security risk the United States has ever faced. AI models are trained by humans. The judgment embedded in pre-training data and especially in expert post-training data largely determines how a model behaves. While emergent behavior exists, a useful approximation is that a model reflects the weighted aggregate of the human judgment distilled into it. Assisting foreign actors—who will naturally prioritize expert tasks aligned with their own interests—to dominate data creation embeds those interests directly into the intelligence layer itself. Once encoded at scale, these interests propagate through every downstream applications that relies on that intelligence. Here’s how we win. First, leverage is in software. China is ahead in hardware for physically intelligent systems. Catching up there is a long and difficult battle. Software, both large language models and robotics models, remains the bottleneck. Advancing the brain (AI models) is the fastest way to increase the usefulness of existing hardware and deployed systems. Second, the U.S. must 100x its investment in structured human judgment. Continued investment in compute and algorithmic efficiency is critical. But that investment is ultimately a bet on very high future inference demand. For that bet to pay off, models must unlock many new capabilities, and in practice the only way to unlock those capabilities is through expert human data. Historically, experts like doctors and lawyers were never incentivized to produce high-quality reasoning data in a machine-verifiable format. There was no reason for a doctor to generate precise, structured simulations of patient interactions, diagnostic reasoning, or treatment tradeoffs. There was no reason for a lawyer to document complex legal reasoning paths in a way that could be programmatically evaluated. AI systems now require exactly this kind of data. The incentive finally exists because this data directly improves systems that operate at massive scale, and experts can be paid well to produce it. Once expert judgment is encoded into models in a structured, verifiable way, it compounds. Those who delay do not just lose time. They lose the ability to catch up. Third, distillation from Chinese labs must be stopped. AI labs must do everything they can to prevent Chinese labs and models from distilling frontier models. Simply calling frontier APIs, or even interacting through UIs, lets Chinese model companies rapidly generate high-quality supervised fine-tuning datasets and close the gap at a fraction of the cost. This method does not put you at the frontier, but it does let you catch up quickly, which is what we saw with DeepSeek. The West significantly overreacted to DeepSeek’s headline capabilities, but underreacted to the underlying dynamic: frontier access itself becomes a training set at a fraction of the cost. Human data platforms also have a duty to help prevent this distillation. Lastly, the U.S.government should set the standard for AI Evaluation that leads to real production usage. AI agents are under-deployed relative to what the technology allows because they are probabilistic systems that require a fundamentally different QA approach than deterministic software. Generic QA is insufficient; safely shipping agents requires explicit evaluation frameworks that assess their full action space. Organizations must clearly define which functions an agent is allowed to perform, how quality is measured for each function, and which domain experts are qualified to judge outcomes. With these frameworks in place, agents can be rigorously tested using structured human data, deployed to production with confidence, and continuously improved over time. The U.S. government should be the first large enterprise to implement rigorous evaluation systems across every function. If the government leads on evaluation-driven deployment, adoption across the private sector accelerates naturally. This is how American workers become more powerful. Each worker operates digital or physical agents that expand their effective output. Recruiting, manufacturing, logistics, and other domains shift toward human judgment overseeing autonomous execution. Reshoring occurs because it becomes economically rational. Work becomes more meaningful. This is a race to determine who controls the intelligence layer of the global economy. And that must be us. 🇺🇸

Ali Ansari

397,098 görüntüleme • 7 ay önce

Dear ICP community, the Internet Computer has now been running strong for 5 years 👏👏👏 Here is a celebratory preview of ICP "cloud engines," the sovereign frontier cloud technology the network shall soon provide from Main points: — Cloud engines enable anyone to spin up their own sovereign frontier cloud. The technology involves an extraordinary inventive step, in which cloud is created from a mathematically secure network of nodes. The nodes run as part of the Internet Computer network ( but are selected and configured by the cloud engine's owner. — The frontier cloud provided by engines is strongly focused on enabling AI agents to build and update online applications and services for us. The world is changing fast, and nearly all new online apps and services are already being built with the help of AI, and thus cloud engines target the future of cloud. — Software hosted on cloud engines is tamperproof, which means that it is immune to infrastructure hacks, because it runs inside a mathematically secure network protocol, rather than on computers directly. This means that AI agents, and those building with them, don't need to have a security team in the loop, or to trust someone else's security team. This is crucial, because in the future, non technical people will demand the freedom to build with full automation — where they just need to issue instructions to AI about what to build, and don't need to worry about anything or anyone else. Of course, apps and services running on engines are also vastly safer from the new breed of hacker being enabled by frontier AI. (The cloud engines themselves are also "tamperproof." Even if a hacker gains physical access to some portion of a cloud engine's nodes, and can make arbitrary changes, the computations and data of the hosted apps and services cannot be corrupted or interrupted so long as the network's fault bounds aren't exceeded. The recent hack of Vercel, a major cloud platform, which gave hackers access to the apps it hosted, provides additional perspective on the importance of this advantage.) — Software hosted on cloud engines is guaranteed to run, so long as a sufficient number of the engine's nodes are running. This means that AI can build applications and services without the need to have a human systems admin team constantly tinkering with the underlying platform to keep it running, which is again crucial, because in the future, non technical people will expect the freedom to use AI to build without the support of others. — New frontier programming language technology, in the form of the Motoko language developed by Caffeine Labs, leverages seminal "orthogonal persistence" technology that unifies program logic and data to deliver further unlocks for AI (Motoko is the first computer language being developed that targets agents that are writing software rather than humans engineers per se). Nowadays, AI can build and update production apps at a prodigious rate, even at the speed of conversation. But it can also make mistakes, and there's a risk that an update it creates might be "lossy" in the sense it causes some transformed data to be lost. Again, in this new world, it's both undesirable and impractical for everyone to have to have a systems admin team on-hand to detect lossy updates and roll them back, but Motoko provides a solution: it can detect new software updates are lossy before they are applied, reducing potentially catastrophic errors by AI to harmless coding retries. — Software hosted on cloud engines is "serverless" but unlike traditional serverless software, directly it directly incorporates data through "orthogonal persistence." Another key purpose is simplify backend software logic and fuel the modeling power of AI by increasing abstraction (sorry for the technical language!!!). Put simply, this enables AI to produce more sophisticated backends, faster, and at dramatically lower costs, as measured by the number AI API tokens consumed during coding. (Tip for the technical: orthogonal persistence is a new paradigm where "the program is the database," and data lives inside program variables, which is possible because it's as if hosted software runs forever in persistent memory). — An expanding database of skills at shall make it possible to develop and directly deploy apps and services to your cloud engines directly from Claude Code, Perplexity, Codex and other AI platforms. Further, your account on can be connected, so that new apps and updates created through conversation automatically appear hosted from your cloud engine. In the future, R&D is going to be very seamless. You converse with AI, and your secure and unstoppable apps or services are created or updated. Cloud engines are designed to directly support this "self-writing cloud" future where we can work hands-free. — Tech sovereignty is becoming a huge issue worldwide, with governments and corporations seeking to create sovereign tech stacks owing to geopolitical tensions. Increasingly, people are realizing that tech provided by foreign nations can come with hidden backdoors and kills switches, from the base platform, right up through hosted apps and services. ICP technology is open source, and those building on ICP using AI own their own source code. When you have the source code, you can verify that there are no backdoors, and when you own the source code thanks to AI, you can update it at will, freeing you from vendor lock-in. But cloud engines take sovereignty much further... — You create a cloud engine by selecting the nodes that will be combined. You can choose the class of nodes used, and their number, but more importantly, you can choose who operates the nodes, and where they are located. Almost any configuration is possible, because the Internet Computer scales the security privileges afforded to hosted software within the network according to configuration (software hosted on cloud engines can directly interoperate with software on other engines and traditional subnets, but base restrictions are applied according to security rules). A cloud engine can be created within a region such as Europe, to comply with regs such as GDPR, or completely within a sovereign state like Switzerland or Pakistan. But cloud engines go further still... — Sovereignty is also about freedom from vendor lock-in. Cloud engines are essentially ICP (Internet Computer Protocol) network configurations, and this means the underlying compute nodes they combine can be swapped out without interrupting their hosted apps and services. This is a big deal. In addition, cloud engines now support nodes that are instances running on Big Tech's clouds, in addition to nodes that are dedicated specialized hardware, as per the Gen I and Gen II nodes that dominate the Internet Computer today. For example, it is possible to have an engine running across different AWS data centers, say, and then reconfigure the engine to run across a mixture of AWS, Google, Azure and Hetzner for even more resilience, without the users of hosted apps and services noticing a thing. That's true freedom. — Sovereign AI is becoming increasingly important too, and cloud engines allow special "AI nodes" to be added to them, so that hosted software can perform inference on hardware provisioned by the owner from a location the owner has selected. Even though the AI nodes are only accessible within the cloud engine, they can still benefit from the forthcoming Internet Intelligence Gateway (IG), which will make it possible to validate inference performed on key frontier open weights LLMs, even when the inference is performed on completely independent AI clouds. When the results of inference are received, this technology can verify that neither the prompt+context (input) nor the inference result (output) have been modified, and that the results were produced by the precise LLM expected. This ensures that AI clouds don't cheat by running inference on cheaper models than are being paid for, and bad actors aren't modifying the inputs or outputs to surreptitiously insert advertising into results, say, or change facts, or insert malware when code is being generated. What's super cool about this technology is the cost of the verification is scalable. A very valuable additional security can be achieved with only 1-2% of extra cost. — Scaling apps and services when they hit capacity limits is another thorny problem that cloud engines help the world address. Engines make scaling possible without rewriting or reconfiguring software. The query workload capacity of hosted software can be horizontally scaled simply by adding new nodes to an engine, and nodes can also be added in geographical proximity to demand. Meanwhile, update workload capacity can first be scaled-up by swapping an engine's nodes out for the next class up, and then when no larger class of node is available, horizontally scaled-out by "splitting" the engine into two, which doubles available capacity. (Technical tip: horizontally scaling update capacity by splitting engines requires multi-canister architectures). — For those who have been following how Caffeine builds apps that can efficiently store large numbers of files, I should mention that apps built on cloud engines will also support the new ICP Blob Storage cloud network (since cloud engines currently have up to about 3 TB of memory, which apps storing large amounts of files can easily exceed). We are also working on allowing blob storage nodes to be added to cloud engines, to enable sovereign mass blob storage within an engine, similarly to how AI nodes can be added currently. — Lastly, but certainly not least, I should mention that cloud engines are multi-blockchain capable, and ready for digital assets, thanks to the clever math at their core. For example, an e-commerce service built on a cloud engine can securely accept and custody stablecoin payments, or a multi-chain DEX could be hosted. Further, engines can support software autonomy (software orchestrated and controlled by other autonomous software, in a decentralized way) and can themselves be orchestrated by SNS technology, and thus run autonomously too. Today, though, the focus is on *mainstream* cloud. This year, the cloud industry will generate approximately one trillion dollars in revenue. That number is already huge, but is expected to grow to two trillion dollars by 2030. After years of continuous development, which have seen more than $500m spent on R&D, the Internet Computer network is now tacking directly toward this mainstream cloud market with cloud engine technology. In their first version, cloud engines are not meant to be a cloud panacea. For example, currently they are not ideal for working with big data. You should use something like DataBricks for that. Cloud engines are carefully targeted at enabling AI to produce traditional online applications and services, including SaaS, in a safer and more productive way, which represents a new market segment with tremendous potential. Of course, DFINITY will continue to work relentlessly to push forward ICP's capabilities, so expect further developments. It's worth mentioning that this cloud segment isn't just about creating new apps and services using AI, it's also about replacing legacy systems and apps built on super expensive SaaS services. Caffeine Labs is working to produce technology (Caffeine Snorkel) that can study an enterprise's legacy systems and app built on SaaS, create replacement systems and apps, and migrate the data, while supporting key stakeholders through the process over email and chat, with full automation. Thus the legacy systems and SaaS markets shall also be addressed by cloud engines. Zooming out, and reasoning in a more metaphysical way, we believe, as we always have, that there is room for a new kind of cloud created by mathematical networks, that provides seminal advances in the fields of security and resilience, as well as true sovereignty and freedom from lock-in. That this same technology, with the help of additional technologies like orthogonal persistence and Motoko, enables AI to build for us without the need for so much oversight, and to create more backend sophistication while consuming fewer AI API tokens, enables ICP to bring game-changing advances to the world. Cloud engines will work synergistically with the Intelligence Gateway, which will enable apps and services running on engines to seamlessly leverage AI, wherever that AI is running, while providing verifiability at extremely low cost for open weights frontier models. We believe that cloud engines represent an inflection point in the storied history of the Internet Computer project, and I'm very proud to be sharing the details with you on the network's fifth birthday 💪 I'll be back with more news soon!!

Dear ICP community, the Internet Computer has now been running strong for 5 years 👏👏👏 Here is a celebratory preview of ICP "cloud engines," the sovereign frontier cloud technology the network shall soon provide from Main points: — Cloud engines enable anyone to spin up their own sovereign frontier cloud. The technology involves an extraordinary inventive step, in which cloud is created from a mathematically secure network of nodes. The nodes run as part of the Internet Computer network ( but are selected and configured by the cloud engine's owner. — The frontier cloud provided by engines is strongly focused on enabling AI agents to build and update online applications and services for us. The world is changing fast, and nearly all new online apps and services are already being built with the help of AI, and thus cloud engines target the future of cloud. — Software hosted on cloud engines is tamperproof, which means that it is immune to infrastructure hacks, because it runs inside a mathematically secure network protocol, rather than on computers directly. This means that AI agents, and those building with them, don't need to have a security team in the loop, or to trust someone else's security team. This is crucial, because in the future, non technical people will demand the freedom to build with full automation — where they just need to issue instructions to AI about what to build, and don't need to worry about anything or anyone else. Of course, apps and services running on engines are also vastly safer from the new breed of hacker being enabled by frontier AI. (The cloud engines themselves are also "tamperproof." Even if a hacker gains physical access to some portion of a cloud engine's nodes, and can make arbitrary changes, the computations and data of the hosted apps and services cannot be corrupted or interrupted so long as the network's fault bounds aren't exceeded. The recent hack of Vercel, a major cloud platform, which gave hackers access to the apps it hosted, provides additional perspective on the importance of this advantage.) — Software hosted on cloud engines is guaranteed to run, so long as a sufficient number of the engine's nodes are running. This means that AI can build applications and services without the need to have a human systems admin team constantly tinkering with the underlying platform to keep it running, which is again crucial, because in the future, non technical people will expect the freedom to use AI to build without the support of others. — New frontier programming language technology, in the form of the Motoko language developed by Caffeine Labs, leverages seminal "orthogonal persistence" technology that unifies program logic and data to deliver further unlocks for AI (Motoko is the first computer language being developed that targets agents that are writing software rather than humans engineers per se). Nowadays, AI can build and update production apps at a prodigious rate, even at the speed of conversation. But it can also make mistakes, and there's a risk that an update it creates might be "lossy" in the sense it causes some transformed data to be lost. Again, in this new world, it's both undesirable and impractical for everyone to have to have a systems admin team on-hand to detect lossy updates and roll them back, but Motoko provides a solution: it can detect new software updates are lossy before they are applied, reducing potentially catastrophic errors by AI to harmless coding retries. — Software hosted on cloud engines is "serverless" but unlike traditional serverless software, directly it directly incorporates data through "orthogonal persistence." Another key purpose is simplify backend software logic and fuel the modeling power of AI by increasing abstraction (sorry for the technical language!!!). Put simply, this enables AI to produce more sophisticated backends, faster, and at dramatically lower costs, as measured by the number AI API tokens consumed during coding. (Tip for the technical: orthogonal persistence is a new paradigm where "the program is the database," and data lives inside program variables, which is possible because it's as if hosted software runs forever in persistent memory). — An expanding database of skills at shall make it possible to develop and directly deploy apps and services to your cloud engines directly from Claude Code, Perplexity, Codex and other AI platforms. Further, your account on can be connected, so that new apps and updates created through conversation automatically appear hosted from your cloud engine. In the future, R&D is going to be very seamless. You converse with AI, and your secure and unstoppable apps or services are created or updated. Cloud engines are designed to directly support this "self-writing cloud" future where we can work hands-free. — Tech sovereignty is becoming a huge issue worldwide, with governments and corporations seeking to create sovereign tech stacks owing to geopolitical tensions. Increasingly, people are realizing that tech provided by foreign nations can come with hidden backdoors and kills switches, from the base platform, right up through hosted apps and services. ICP technology is open source, and those building on ICP using AI own their own source code. When you have the source code, you can verify that there are no backdoors, and when you own the source code thanks to AI, you can update it at will, freeing you from vendor lock-in. But cloud engines take sovereignty much further... — You create a cloud engine by selecting the nodes that will be combined. You can choose the class of nodes used, and their number, but more importantly, you can choose who operates the nodes, and where they are located. Almost any configuration is possible, because the Internet Computer scales the security privileges afforded to hosted software within the network according to configuration (software hosted on cloud engines can directly interoperate with software on other engines and traditional subnets, but base restrictions are applied according to security rules). A cloud engine can be created within a region such as Europe, to comply with regs such as GDPR, or completely within a sovereign state like Switzerland or Pakistan. But cloud engines go further still... — Sovereignty is also about freedom from vendor lock-in. Cloud engines are essentially ICP (Internet Computer Protocol) network configurations, and this means the underlying compute nodes they combine can be swapped out without interrupting their hosted apps and services. This is a big deal. In addition, cloud engines now support nodes that are instances running on Big Tech's clouds, in addition to nodes that are dedicated specialized hardware, as per the Gen I and Gen II nodes that dominate the Internet Computer today. For example, it is possible to have an engine running across different AWS data centers, say, and then reconfigure the engine to run across a mixture of AWS, Google, Azure and Hetzner for even more resilience, without the users of hosted apps and services noticing a thing. That's true freedom. — Sovereign AI is becoming increasingly important too, and cloud engines allow special "AI nodes" to be added to them, so that hosted software can perform inference on hardware provisioned by the owner from a location the owner has selected. Even though the AI nodes are only accessible within the cloud engine, they can still benefit from the forthcoming Internet Intelligence Gateway (IG), which will make it possible to validate inference performed on key frontier open weights LLMs, even when the inference is performed on completely independent AI clouds. When the results of inference are received, this technology can verify that neither the prompt+context (input) nor the inference result (output) have been modified, and that the results were produced by the precise LLM expected. This ensures that AI clouds don't cheat by running inference on cheaper models than are being paid for, and bad actors aren't modifying the inputs or outputs to surreptitiously insert advertising into results, say, or change facts, or insert malware when code is being generated. What's super cool about this technology is the cost of the verification is scalable. A very valuable additional security can be achieved with only 1-2% of extra cost. — Scaling apps and services when they hit capacity limits is another thorny problem that cloud engines help the world address. Engines make scaling possible without rewriting or reconfiguring software. The query workload capacity of hosted software can be horizontally scaled simply by adding new nodes to an engine, and nodes can also be added in geographical proximity to demand. Meanwhile, update workload capacity can first be scaled-up by swapping an engine's nodes out for the next class up, and then when no larger class of node is available, horizontally scaled-out by "splitting" the engine into two, which doubles available capacity. (Technical tip: horizontally scaling update capacity by splitting engines requires multi-canister architectures). — For those who have been following how Caffeine builds apps that can efficiently store large numbers of files, I should mention that apps built on cloud engines will also support the new ICP Blob Storage cloud network (since cloud engines currently have up to about 3 TB of memory, which apps storing large amounts of files can easily exceed). We are also working on allowing blob storage nodes to be added to cloud engines, to enable sovereign mass blob storage within an engine, similarly to how AI nodes can be added currently. — Lastly, but certainly not least, I should mention that cloud engines are multi-blockchain capable, and ready for digital assets, thanks to the clever math at their core. For example, an e-commerce service built on a cloud engine can securely accept and custody stablecoin payments, or a multi-chain DEX could be hosted. Further, engines can support software autonomy (software orchestrated and controlled by other autonomous software, in a decentralized way) and can themselves be orchestrated by SNS technology, and thus run autonomously too. Today, though, the focus is on mainstream cloud. This year, the cloud industry will generate approximately one trillion dollars in revenue. That number is already huge, but is expected to grow to two trillion dollars by 2030. After years of continuous development, which have seen more than $500m spent on R&D, the Internet Computer network is now tacking directly toward this mainstream cloud market with cloud engine technology. In their first version, cloud engines are not meant to be a cloud panacea. For example, currently they are not ideal for working with big data. You should use something like DataBricks for that. Cloud engines are carefully targeted at enabling AI to produce traditional online applications and services, including SaaS, in a safer and more productive way, which represents a new market segment with tremendous potential. Of course, DFINITY will continue to work relentlessly to push forward ICP's capabilities, so expect further developments. It's worth mentioning that this cloud segment isn't just about creating new apps and services using AI, it's also about replacing legacy systems and apps built on super expensive SaaS services. Caffeine Labs is working to produce technology (Caffeine Snorkel) that can study an enterprise's legacy systems and app built on SaaS, create replacement systems and apps, and migrate the data, while supporting key stakeholders through the process over email and chat, with full automation. Thus the legacy systems and SaaS markets shall also be addressed by cloud engines. Zooming out, and reasoning in a more metaphysical way, we believe, as we always have, that there is room for a new kind of cloud created by mathematical networks, that provides seminal advances in the fields of security and resilience, as well as true sovereignty and freedom from lock-in. That this same technology, with the help of additional technologies like orthogonal persistence and Motoko, enables AI to build for us without the need for so much oversight, and to create more backend sophistication while consuming fewer AI API tokens, enables ICP to bring game-changing advances to the world. Cloud engines will work synergistically with the Intelligence Gateway, which will enable apps and services running on engines to seamlessly leverage AI, wherever that AI is running, while providing verifiability at extremely low cost for open weights frontier models. We believe that cloud engines represent an inflection point in the storied history of the Internet Computer project, and I'm very proud to be sharing the details with you on the network's fifth birthday 💪 I'll be back with more news soon!!

dom | icp

320,117 görüntüleme • 4 ay önce

Wuthering Waves, and what's next... A visit to Kuro Games headquarters an exclusive interview with CEO Solon by YTN Team Full Video: Q: Looking back, what first motivated the team to take on such an ambitious project? A: From the very beginning, we realized something important: when a goal feels difficult but the result is exciting, people’s eyes light up. That moment tells us the challenge is worth pursuing. We genuinely enjoy tackling hard problems, because we believe meaningful achievements only come from doing things that are difficult but right. Q: Your early projects gained attention very quickly. How did that affect the team? A: When our first product launched, the response exceeded expectations. At the time, it almost made us believe we already had everything figured out. But that confidence was quickly tested. We lacked experience in long-term content planning, sustainable version updates, and large-scale operations. Many issues only became apparent after launch, forcing us to learn by solving problems in real time. Q: How did those early lessons shape Punishing: Gray Raven? A: In hindsight, many early versions of our games didn’t fully reach their potential. Those shortcomings became invaluable lessons. When developing Punishing: Gray Raven, we consciously avoided repeating those mistakes. We studied player ecosystems, observed how other successful titles operated, and anticipated feedback before it arrived. That preparation laid the groundwork for a project that has now sustained itself for nearly six years. Q: When did the concept for Wuthering Waves take shape? A: With Wuthering Waves, our goal was very clear from the start: to create something players had never experienced before. We drew inspiration from works like Death Stranding, Annihilation, and outstanding domestic animation. Themes such as resonance, abstract spaces, supernatural phenomena, and strong audiovisual expression naturally pointed us toward an open-world action game. Q: Sound and “resonance” play a major role in the worldbuilding. Why was that important? A: We wanted sound to be more than atmosphere, it needed to be a narrative force. The idea of a world-altering resonance, a low-frequency “lament” capable of reshaping reality, became the foundation of the setting. From there, the world, its conflicts, and its powers gradually took form. Kuro aims to create concepts players have never experienced before. Inspirations include: - Supernatural themes. - Strong mechanical and emotional imagery. - Music as a storytelling medium. Q: How do you ensure cohesion between story, gameplay, and world design? A: Alignment is critical. Story, environment, and gameplay must reinforce each other. That requires long-term planning, especially when multiple versions are developed in parallel. Early coordination allows level design, systems, and IP development to inspire one another, which improves both efficiency and creative momentum. The goal is to avoid disconnected content between versions and maintain narrative and gameplay cohesion. Q: What does Version 3.0 represent for Wuthering Waves? A: Version 3.0 is a major milestone. It introduces an entirely new region, new characters, upgraded combat systems, and a significantly improved open-world traversal experience. Every version aims to deliver the best experience possible while continuing to surprise players. Q: How does the team handle player feedback? A: We monitor feedback through community data, sentiment analysis, and post-version reviews. Both positive and negative responses are carefully analyzed and reflected in future updates. Continuous iteration is essential to long-term success. Q: Early story reception was mixed. How did the team respond? A: We recognized that narrative presentation needed improvement. By optimizing production pipelines and increasing investment, later versions achieved much stronger storytelling clarity and delivery. Players can now more clearly feel the depth, direction, and long-term potential of the world. Q: Kuro Games is known for polish. How central is that philosophy? A: We believe every visible part of the game must meet high internal standards, like - Visual effects - Skills - Environmental details -Music and audio design or promotional content must meet a high standard. We want players to feel the identity and power of Wuthering Waves both inside and outside the game. We know that early story reception was weaker, but significant effort has been invested in: - Clearer storytelling - Stronger presentation - More emotionally impactful performances and scenes. Q: How do community events factor into long-term development? A: Long-term IP growth depends on emotional connection. Community events, both online and offline, help us understand player expectations while strengthening that bond. It’s a two-way relationship. Q: Looking ahead, what is Kuro Games’ long-term vision? A: Our goal is to become a globally influential action-game developer. Upcoming projects (e.g., Nami, still in development) are focused on: - Delivering unique action combat experiences. - Creating something clearly distinct within the action game genre. The story of Wuthering Waves is planned years in advance, with major narrative arcs already outlined. Q: Finally, how would you describe Kuro Games’ current phase? A: From Zhuhai to Guangzhou, we’re in a strong upward phase. Our growth is driven by a commitment to meaningful, high-quality experiences. Staying true to that original intent is what will continue to guide us forward. #WutheringWaves #punishinggrayraven

Wuthering Waves, and what's next... A visit to Kuro Games headquarters an exclusive interview with CEO Solon by YTN Team Full Video: Q: Looking back, what first motivated the team to take on such an ambitious project? A: From the very beginning, we realized something important: when a goal feels difficult but the result is exciting, people’s eyes light up. That moment tells us the challenge is worth pursuing. We genuinely enjoy tackling hard problems, because we believe meaningful achievements only come from doing things that are difficult but right. Q: Your early projects gained attention very quickly. How did that affect the team? A: When our first product launched, the response exceeded expectations. At the time, it almost made us believe we already had everything figured out. But that confidence was quickly tested. We lacked experience in long-term content planning, sustainable version updates, and large-scale operations. Many issues only became apparent after launch, forcing us to learn by solving problems in real time. Q: How did those early lessons shape Punishing: Gray Raven? A: In hindsight, many early versions of our games didn’t fully reach their potential. Those shortcomings became invaluable lessons. When developing Punishing: Gray Raven, we consciously avoided repeating those mistakes. We studied player ecosystems, observed how other successful titles operated, and anticipated feedback before it arrived. That preparation laid the groundwork for a project that has now sustained itself for nearly six years. Q: When did the concept for Wuthering Waves take shape? A: With Wuthering Waves, our goal was very clear from the start: to create something players had never experienced before. We drew inspiration from works like Death Stranding, Annihilation, and outstanding domestic animation. Themes such as resonance, abstract spaces, supernatural phenomena, and strong audiovisual expression naturally pointed us toward an open-world action game. Q: Sound and “resonance” play a major role in the worldbuilding. Why was that important? A: We wanted sound to be more than atmosphere, it needed to be a narrative force. The idea of a world-altering resonance, a low-frequency “lament” capable of reshaping reality, became the foundation of the setting. From there, the world, its conflicts, and its powers gradually took form. Kuro aims to create concepts players have never experienced before. Inspirations include: - Supernatural themes. - Strong mechanical and emotional imagery. - Music as a storytelling medium. Q: How do you ensure cohesion between story, gameplay, and world design? A: Alignment is critical. Story, environment, and gameplay must reinforce each other. That requires long-term planning, especially when multiple versions are developed in parallel. Early coordination allows level design, systems, and IP development to inspire one another, which improves both efficiency and creative momentum. The goal is to avoid disconnected content between versions and maintain narrative and gameplay cohesion. Q: What does Version 3.0 represent for Wuthering Waves? A: Version 3.0 is a major milestone. It introduces an entirely new region, new characters, upgraded combat systems, and a significantly improved open-world traversal experience. Every version aims to deliver the best experience possible while continuing to surprise players. Q: How does the team handle player feedback? A: We monitor feedback through community data, sentiment analysis, and post-version reviews. Both positive and negative responses are carefully analyzed and reflected in future updates. Continuous iteration is essential to long-term success. Q: Early story reception was mixed. How did the team respond? A: We recognized that narrative presentation needed improvement. By optimizing production pipelines and increasing investment, later versions achieved much stronger storytelling clarity and delivery. Players can now more clearly feel the depth, direction, and long-term potential of the world. Q: Kuro Games is known for polish. How central is that philosophy? A: We believe every visible part of the game must meet high internal standards, like - Visual effects - Skills - Environmental details -Music and audio design or promotional content must meet a high standard. We want players to feel the identity and power of Wuthering Waves both inside and outside the game. We know that early story reception was weaker, but significant effort has been invested in: - Clearer storytelling - Stronger presentation - More emotionally impactful performances and scenes. Q: How do community events factor into long-term development? A: Long-term IP growth depends on emotional connection. Community events, both online and offline, help us understand player expectations while strengthening that bond. It’s a two-way relationship. Q: Looking ahead, what is Kuro Games’ long-term vision? A: Our goal is to become a globally influential action-game developer. Upcoming projects (e.g., Nami, still in development) are focused on: - Delivering unique action combat experiences. - Creating something clearly distinct within the action game genre. The story of Wuthering Waves is planned years in advance, with major narrative arcs already outlined. Q: Finally, how would you describe Kuro Games’ current phase? A: From Zhuhai to Guangzhou, we’re in a strong upward phase. Our growth is driven by a commitment to meaningful, high-quality experiences. Staying true to that original intent is what will continue to guide us forward. #WutheringWaves #punishinggrayraven

Naru

72,974 görüntüleme • 9 ay önce

Just in $AMD Anush "Speed is the moat"|ROCm🎙️ In the race to define the future of AI, what's the one advantage that truly lasts? It's not proprietary tech, argues Anush Elangovan Elangovan, VP of AI Software at AMD , but the sustainable speed of innovation. He explains why AMD is rejecting the "walled garden" model for its open source ROCm stack, betting that an open community flywheel is the key to victory. Listen to understand how this open strategy is designed to out-innovate closed systems by empowering developers to solve everything from frontier-model challenges to the mundane, everyday problems that define the "last mile" of AI. AMD ROCm Software: Part 1 Transcript [00:00:00] Andrew Zigler: Joining me is Anush Elangovan, VP of AI software at AMD. And when people talk about AI compute, the conversation often stops at hardware specs, but it's more than just physical chips that win the game. It's also the software ecosystems supporting them. [00:00:18] Andrew Zigler: The prevailing strategy in the industry has been to build something like a walled garden. You know, something closed, proprietary locks, developers in. But AMD is betting on an entirely different play, open source acceleration, and with rock, their open source AI software stack. AMD is building not just hardware parity, but an innovation flywheel that's powered by the community with interoperability and the freedom to scale without all of that pesky lockin. [00:00:48] Andrew Zigler: And in this world, speed is your moat and how fast you can innovate while your platform remains open, flexible, and standardize across all of its applications. That's what we're gonna explore [00:01:00] today. So Anush, I'm really excited to have you here. Welcome to Dev Interrupted. [00:01:04] Anush Elangovan: Thanks for having me. Uh, super excited to chat about it. [00:01:07] Andrew Zigler: Amazing. Well, let's go ahead and dive right in with kind of what I laid it out with in the beginning, the idea of the moat and it being about speed. I wanna unpack that a bit because that came from you when you and I first spoke. And I, and I want to know, you know, how do you define speed inside of AMD beyond just things like hardware, benchmarks. [00:01:27] Anush Elangovan: Yeah, that's a very good question. So when we typically talk about speed, everyone's like, Hey, hardware benchmark specs, right? Like, uh, memory bandwidth or, or flops. And that is one important part of it, uh, AMD does very well. With that, we do have, a, a very good history of executing on that axis. [00:01:47] Anush Elangovan: But when I say speed is the moat, it is about, uh, how we prepare, how we build the muscle to run the race for a long time and run it fast. And it is [00:02:00] not about a single point in time that you've, you've beat some you know, benchmark and, and you declare victory. It's about building the ability to consistently develop and deliver. [00:02:13] Anush Elangovan: Both hardware and software innovation at scale and do it fast, right? Like, you know, we we're increasingly getting to a point where models come out and they're, uh, you know, a year or two ago it was like, Hey, they work on AMD on day zero, which is great, but now they are performing on AMD the day it releases, right? [00:02:32] Anush Elangovan: So, what does it take to Prefetch where the industry is going? Be prepared to intercept. At that point is what you know, I, I refer to as you know, the, the speed factor in, in creating this mode, right? And the mode is just shed all things that hold you back and run as fast as you can. [00:02:53] Anush Elangovan: Uh, because the pace of innovation that is, uh, being seen in, in AI [00:03:00] industries is just. Amazing. Right? And it's like, it's transformational at at how you generate electricity. It's transformational as at how you build data centers. It's transformational at how you deploy compute, networking. It's transformational at what kind of use cases you, you know, uh, use AI for. [00:03:17] Anush Elangovan: Uh, and for that, you need to be prepared to, see what comes tomorrow and be prepared to run the race tomorrow. [00:03:23] Andrew Zigler: Yeah, it's a really great perspective because it highlights that it's not just like a checkpoint that you run through. I like how you called out, like it's not just hitting that benchmark or being the best in class at that moment, in that snapshot, it's about having a. The throughput and about having that dedication to the idea and continuing to deliver on it. [00:03:43] Andrew Zigler: It's not just crossing the threshold, but it's also being the engine. And that's what, that's what protects a business. That is the moat, because the moat is that innovation layer, the faster and more, uh, future forward. That you can work and think, [00:04:00] you know, the better. Uh, we, we talk a lot about like future forward work styles. [00:04:04] Andrew Zigler: Like what are the things I could be doing right now today that are gonna be like, way more useful tomorrow? Let, let's abandon those, workflows that are older and that kind of like, that translates into. An advantage when you work that way. You know, what kind of things have you learned working with, uh, like across all spectrums of people who would use ROCm, right? [00:04:23] Andrew Zigler: You have like the developers, but then you also have the enterprises and you have this large span of adoptees, right? So what is the, what does that look like that you learn? [00:04:32] Anush Elangovan: Yeah, so, so the way I look at it is there are gonna be pockets of different, uh, you know, cadences, right? Like, so people who are deploying in enterprises, for example, right? The validation and how long it takes for them to deploy an LLM that's secure. It's, with guardrails, et cetera, maybe longer. [00:04:52] Anush Elangovan: but you still have to go through the process and you have to be prepared to like, walk that walk to deploy an enterprises. That doesn't mean it's [00:05:00] not fast, that's as fast as you can do for that industry, right? And if you are deploying AI in healthcare, right, it's, it's got its own, uh, cycle. [00:05:07] Anush Elangovan: but in each one of these, you want to see how, like, go down to the essence of what is it that you actually have to do. And, you know, I, I, I like how you framed it. It's like it's, you shed your prior assumptions of how things are done, right. And, and you kind of build up from a, uh, first principles, uh, approach to say, this is how I could use AI to unlock, whatever I'm doing. [00:05:33] Anush Elangovan: And, and, some of it, you know, it's good to really step back and look at. Just question every part of it, right? Like right now you're getting chat GPT and, Gemini competing for like, math, olympiads and, and, uh, college, uh, reasoning, uh, tests. Right? And, and those are like that, that is amazing and increasingly like complex tasks that they're trying to do. [00:05:58] Anush Elangovan: But there may also be like. [00:06:00] More mundane things that AI could, could get applied to. Right? And, and so when we think about shedding old ways, you wanna shed it not just in like the tip of the spear. It's like, you know, I'm gonna see what's the frontier model. It's also, it could be something as simple as. [00:06:18] Anush Elangovan: How do you choose a, a movie, uh, you know, like a recommendation system, right? Or, or, uh, an automated, uh, flight, uh, rebooking system. So the moment, you know, your flight is late, uh, right now it's a notification, right? It's like, oh, you got a text message saying your flight's late. And I got that like three times this week. [00:06:38] Anush Elangovan: But anyway, uh, and, and, and, and, I was just like, okay, so if I were to rethink this. All this MCPs that we have that should be hooked up into an MCP that says, your flight's delayed. Here are your options. If you want, you know, these are the paid options. Yeah. Here are the free options. This will get you back into your you know, Toronto airport [00:07:00] tonight. [00:07:00] Anush Elangovan: Or if you stay, here's a hotel plus this, plus this, plus. It's just like, go ahead is all I should say. Versus now I'm like, okay, can someone, you know, can I call a travel agent? Can I do this? Can I go online and log into And you know, so we gotta fundamentally rethink even those like small, nuances of, things that we do that can be automated out and AI is really, really good at doing something like this, right? Maybe I just explained an AI startup idea right now. Somebody should just start that. [00:07:29] Andrew Zigler: I think you did. Yeah, you definitely did. Someone, one of our listeners is definitely going to lift that off of you. I, I, I, you know, I hate being on the receiving end of those. You feel a little helpless and then you have to like, follow the whole flow. So I know what you mean. Like I, I like how you called out that the build and this like. [00:07:45] Andrew Zigler: Where speed is your moat and the innovation layer is protecting you, is what makes you better than your competitors. How you scale that and you bring that to market. So by understanding the problems that you're solving, uh, throwing away those older assumptions, but also [00:08:00] recognizing that like. We're building every single day, new things and new ways of using stuff that we're still figuring out the implications of. [00:08:08] Andrew Zigler: And so when you have a lot of velocity and you're introducing a lot of new ideas, and maybe you have that workflow now that automatically rebook your flight off of your late flight text message, and uh, I know I would certainly use it, but you know, what kind of philosophies guide the way that y'all think about building this ecosystem to manage that stability while letting folks. [00:08:29] Andrew Zigler: Play with the speed and the assumptions and the airplane re bookings. [00:08:34] Anush Elangovan: so, so I think, you know, we need to peel one layer down, right? and the philosophy is, Hey, we, we just discovered electricity, right? And you know what we're gonna do? We are gonna make motors, uh, or dynamos, right? Like engines. Uh, sure. We don't know if it's gonna be a Ferrari that you're gonna make, or it's a a a a dump truck. [00:08:57] Anush Elangovan: That's good for doing this. But let's [00:09:00] let, which is also required, right? You need a dump truck. You need a garbage truck. And, [00:09:04] Andrew Zigler: Yeah. You need the [00:09:04] Anush Elangovan: course you need, uh, a Ferrari for a midlife crisis, right? So, [00:09:09] Andrew Zigler: precisely. [00:09:10] Anush Elangovan: But, but my, uh, point is what do we build next? And, uh, and this is what I meant by like, okay, let's, let's take those baby steps to build the. [00:09:20] Anush Elangovan: Infrastructure that's required that we know we'll have to use, right? So, so if I just discovered electricity, okay, great. Now one, how do I save this electricity and how do I use it? So there's battery technology, so you need to do something like that, right? Like so. But then you also want to make it into an actionable thing. [00:09:37] Anush Elangovan: You want to make it for like automobiles, or you wanna use it for, you know, powering, uh, entire cities. So it is that transformational. So, uh, AI is that transformational. So, if you distill down, it'll, it'll come down to how do we think about, what we can do with this this fundamental technology that, We may not be aware of what it [00:10:00] is gonna unlock next, but at least you know the next step is clear, right? It's like a dense fog, you know, it's gonna be like, it, it's the right path. You see the light, but it's kind of like out there and, and the steps you're taking are concrete and you're like, okay, this is good. [00:10:16] Anush Elangovan: I, this is better than where I was or where we were. So we are moving forward. So you can build with the. Intuition from what you see in the short term and a tactical view, but towards what you think the future is gonna be. [00:10:28] Andrew Zigler: Right. You almost like we're all in this like fog of war, right? And like you said, you're reaching out and you're trying to step through it. You could think of it too, as like you're in the dark and your hands are up in front of you and you know that. You're, you're not gonna run your face into a wall because your hands are out in front of you, but you're not gonna maybe do much better than that. [00:10:45] Andrew Zigler: So that's kind of like, I think the eco, the, the industry, the world that we find ourselves in, uh, and we all have to, then this becomes the power of an ecosystem, of a group of people working together to create that layer of, [00:11:00] uh, of establishing the [00:11:01] Anush Elangovan: exactly. And I, I, I just, instead of, you know, saying fog of war I describe it as like, you're in this. Beautiful valley with like a morning, uh, fog that's in. You can smell the flowers. You, you hear the birds. You are like, okay, it's, we are in like, uh, utopian paradise and yes, I just need to like, continue the walk, right? [00:11:24] Anush Elangovan: and then move forward with that, conviction that you're in the right spot. [00:11:27] Andrew Zigler: Yeah. So let's talk about that ecosystem world. This nice, I love how you describe it, this grassy side of a hill in the morning that's covered in some mist and maybe we can't see 30 feet in one direction, but it sure is a beautiful hill and it smells nice. And so we're all here. And why is, in that world, why is. [00:11:44] Andrew Zigler: You know, open source, their strategic advantage that y'all are going for in the AI hardware market. And, and then how does like ROCm turn that into wins for people within that ecosystem? [00:11:56] Anush Elangovan: you know, the, the way we look at it is this, is kind of like how I view [00:12:00] AI and the ecosystem, right? But, but it is for everyone to enjoy. Uh, and so we do want to make sure that. You know, it is, uh, beneficial for everyone. [00:12:09] Anush Elangovan: The ecosystem can come in and, and innovate. It's an open innovation engine. and uh, it is very different from, you know, having a walled garden with, Hey, only I know how to do this and I'm gonna do it and throw it over the fence and you can use it or keep walking, right? So we'd like to be good citizens that way, but also. [00:12:30] Anush Elangovan: Uh, it is self-fulfilling in a way, right? Like it, the, the pace at which we innovate with open source is unmatched. Like, you know, our serving engines are like VLLM and, and sg l. Those things, uh, those frameworks are like super, super aggressive in terms of how fast they come out with features and how fast they can you know, get performant models out. [00:12:52] Anush Elangovan: And that compared with what, uh, you'd get from, you know, the likes of like T-R-T-L-L-M or something is always lagging, right? Because you [00:13:00] just can't keep up with you know, 200 commits a week just on one particular model to get that model really performant [00:13:06] Andrew Zigler: And, and, and in that world where, you know, everyone can enjoy the winds of this, what kind of customer stories or innovation stories have really stood out to you and excite you about building and creating this place for developers? [00:13:19] Anush Elangovan: Yeah. So I think the parts that are super exciting for me are when when we get to see a customer that is first skeptical. Then they start a little like, okay, fine, we'll give you a chance. Uh, we do a simple, uh, POC and then they're like, huh, this seems to work. Yeah, we told you it works. [00:13:42] Anush Elangovan: You don't have to change one line of code. Really? Yes, no need to change one line of code. Okay, let's try a production workload. So then they try it. Oh, you're more performant than the competition. Yes. We're more performant than, than the competition. So how much does it cost? And we're like, oh, it's your TCO is better with, uh, [00:14:00] AMD. [00:14:00] Anush Elangovan: So again, they're like, wow, okay, good. So now how do we deploy at scale? And then we go deploy it at scale. And when they give a thumbs up on that and they say, this is good, right? That's when you know, you, you see it go full circle from like, oh, we, we've never heard about AMD to like actually deploy to tens of thousands of GPUs In the order of a few months, right? It, it, it really is fascinating to see and very exciting and invigorating to [00:14:28] Andrew Zigler: Yeah. At like a great exposure to a lot of interesting problems. And, and then people using the infrastructure, the, the technology available to solve those problems. Really specific problems by the way, that's often why they're bringing their data and AI to it, uh, is because it is really specific and important for them. [00:14:45] Andrew Zigler: And there's a, a lot I think that other engineering orgs can learn and even emulate from AMD's success and, and having this open source ecosystem and it causing this acceleration within. You [00:15:00] know, uh, customers and enterprises that use and adopt the tools and, and, and that creates an advantage. And that goes back to why we're talking and like the real thesis of our conversation today. [00:15:10] Andrew Zigler: So how do you think engineering leaders that are listening to this and obviously tapping into this great success AMD has from an open source flywheel, how do you think other, other folks building in the same space can foster that open, first, that open source oriented culture in order to, you know, accelerate their innovation goals? [00:15:29] Anush Elangovan: Yeah, that's a very good question. So the startup that um, was acquired by AMD we, we built, I mean, we started off doing iot stuff and you know, smart ring and all that, right? But in the, the end of like, uh, and not the end, the last six years of the company was building ML compilers. [00:15:47] Anush Elangovan: And ml, ML compilers are like super, uh, complicated, sophisticated, advanced algorithms, dah, dah, dah. but it was all open source, right? So our VCs were like, wait, what do you mean your core [00:16:00] IP is open source? And um, the speed is the moat applied even then, right? It was just like, yes, if you have an idea that. [00:16:08] Anush Elangovan: Because someone saw this idea that you are, they're gonna be able to catch up, then you probably have the wrong idea anyway. But if they are, you know, you execute and they're gonna catch up, that you should assume they're gonna catch up. Right? So you gotta move forward. So keeping it open source is super important. [00:16:25] Anush Elangovan: But also to your question on like, you know, the learnings from an AMD standpoint, right? If there are, hard problems, I'd say dig in and work through it, right? Like there's no way but through it, right? That should be the simple mentality. And more, uh, frequently than not. you'll see that you'll just make it through in a, in, in good form. [00:16:52] Anush Elangovan: But if you doubt it and you're like, oh, I don't know if I should commit, if I'm, I, you know, what should just commit to do the right thing [00:17:00] every step, right? Every step, and just keep taking one step in front of the other. And in no time you'll see that you'll be running. Right. And, and yes, the first few steps will be like, yeah, everyone's complaining about your software quality. [00:17:15] Anush Elangovan: Everyone's complaining about this and that, and it doesn't work. And, and a few steps in, you know, you get, you get the hang of all the complaints that are coming in. You get the feedback loop. You're like, okay, what, what are you prioritizing again? One step in front of the other, right? You just keep knocking that out and then you get to a point where you're, it just becomes second nature, right? To do the, to do the right thing. And, and then yes, if someone gives you two options, you'll be like, fine. This is, uh, you know, there's always the resource trade off. There's always a human capital trade off, but what's the right thing to do? of course, I, I'm pragmatic about what we choose, but, but if the right thing for your long-term success is dig in, go first, principles, make it [00:18:00] happen. [00:18:00] Anush Elangovan: Well. Then just go for that. There's, there is no shortcut to [00:18:04] Andrew Zigler: acknowledging, you know, how it aligns with your mission, your core company goals, and what you're looking to achieve. And, and I, I love how you rightfully called out that in the open source world and you know, you have your technology that you've built, what you think is your moat upon, right? [00:18:22] Andrew Zigler: It's your code and, and to open source that, or to just make it where anyone could peer in is, you know. Scary in one regard, but two, it just kind of feels like you're handing away your throne room in some kind of sense, a very direct feeling sense. But the ultimately, you were really right to call out, and this is something I think about all the time, that the real power there is still the speed This the speed. [00:18:42] Andrew Zigler: That was the moat at the beginning of our conversation. It's the speed in combination with your. Very specific domain understanding of what you're building and what you're creating, and your new role as the steward of that world and how people plug into it, which [00:19:00] has frankly, a lot more influence and power than lording over a closed. [00:19:04] Andrew Zigler: You know, repository or an ecosystem, and like you said, like throwing things over the wall. Sure. There, there might be people always on the other side of that wall, but you're not gonna have a great connection with them. You're not gonna be able to really clearly understand them. I, I like your metaphor of the side of the field of the mountain a lot more. [00:19:23] Andrew Zigler: But, but in the, in this world, you know, where. That speed is, is the power and, and open source is just one way that you can harness that speed to get really far ahead and to innovate. , There's other parts of this equation that you can be experimenting with too, and I'd love to pick your brain about them as a software leader and, and, and one of them is about looking forward and kind of understanding that future that we're all building towards and beyond today's models and hardware. [00:19:48] Andrew Zigler: You know, what do you see as the next major bottleneck or opportunity in the AI compute space? As, as you know, enterprises and folks start to get a little more mature about what's available to [00:20:00] them. [00:20:00] Anush Elangovan: Yeah, I think, the bottleneck and opportunity is, uh, what I'd call, call walking the last mile of ai. Right. Uh, and like I I, I gave you an example, uh, previously, but, but it's similar to that. It's like there are cases where Humans have so many, uh, things to do in your day. You know, like the, if we sit down and actually had a customer focus like, okay, these customers lives, I'm gonna save four hours of this customer's life. And if you actually sit down and look at all of that, it'll be. Easily automatable, easily you know, uh, applicable, uh, for ai, right? [00:20:39] Anush Elangovan: Like, but then making it happen is gonna take a little bit, right? It's like maybe it's, uh, paying your utility bill, right? Or something like that, right? Or, or, your healthcare explanation of benefits. Uh, like, I'm sure you get an explanation of benefits, and I'm like, I, I don't even know what that thing is. [00:20:55] Anush Elangovan: It's just like EOB and like. [00:20:57] Andrew Zigler: it's a big, a big old PDF. Yeah, [00:21:00] exactly. [00:21:01] Anush Elangovan: Like, like, I'm like great straight to the, uh, shredder, right? And but that could be, you know, automated with the ai, right? It, it, it'd be like, Hey, the summary of this thing is you went and visited this day. Everything is okay. Everything is paid for, so don't worry, it's not a bill. [00:21:17] Anush Elangovan: That again, the same, uh, thing, but the sense of what that information overload is could be. Digested by ai, uh, accumulated over time and retrieved when you need it. Like, I don't, I actually don't even need to know this EOB right now, unless of course, whenever I need to know it, that maybe, you know, like for some benefits I need to figure out what do, what did I do over the past year and how do I apply it? Source:

Just in $AMD Anush "Speed is the moat"|ROCm🎙️ In the race to define the future of AI, what's the one advantage that truly lasts? It's not proprietary tech, argues Anush Elangovan Elangovan, VP of AI Software at AMD , but the sustainable speed of innovation. He explains why AMD is rejecting the "walled garden" model for its open source ROCm stack, betting that an open community flywheel is the key to victory. Listen to understand how this open strategy is designed to out-innovate closed systems by empowering developers to solve everything from frontier-model challenges to the mundane, everyday problems that define the "last mile" of AI. AMD ROCm Software: Part 1 Transcript [00:00:00] Andrew Zigler: Joining me is Anush Elangovan, VP of AI software at AMD. And when people talk about AI compute, the conversation often stops at hardware specs, but it's more than just physical chips that win the game. It's also the software ecosystems supporting them. [00:00:18] Andrew Zigler: The prevailing strategy in the industry has been to build something like a walled garden. You know, something closed, proprietary locks, developers in. But AMD is betting on an entirely different play, open source acceleration, and with rock, their open source AI software stack. AMD is building not just hardware parity, but an innovation flywheel that's powered by the community with interoperability and the freedom to scale without all of that pesky lockin. [00:00:48] Andrew Zigler: And in this world, speed is your moat and how fast you can innovate while your platform remains open, flexible, and standardize across all of its applications. That's what we're gonna explore [00:01:00] today. So Anush, I'm really excited to have you here. Welcome to Dev Interrupted. [00:01:04] Anush Elangovan: Thanks for having me. Uh, super excited to chat about it. [00:01:07] Andrew Zigler: Amazing. Well, let's go ahead and dive right in with kind of what I laid it out with in the beginning, the idea of the moat and it being about speed. I wanna unpack that a bit because that came from you when you and I first spoke. And I, and I want to know, you know, how do you define speed inside of AMD beyond just things like hardware, benchmarks. [00:01:27] Anush Elangovan: Yeah, that's a very good question. So when we typically talk about speed, everyone's like, Hey, hardware benchmark specs, right? Like, uh, memory bandwidth or, or flops. And that is one important part of it, uh, AMD does very well. With that, we do have, a, a very good history of executing on that axis. [00:01:47] Anush Elangovan: But when I say speed is the moat, it is about, uh, how we prepare, how we build the muscle to run the race for a long time and run it fast. And it is [00:02:00] not about a single point in time that you've, you've beat some you know, benchmark and, and you declare victory. It's about building the ability to consistently develop and deliver. [00:02:13] Anush Elangovan: Both hardware and software innovation at scale and do it fast, right? Like, you know, we we're increasingly getting to a point where models come out and they're, uh, you know, a year or two ago it was like, Hey, they work on AMD on day zero, which is great, but now they are performing on AMD the day it releases, right? [00:02:32] Anush Elangovan: So, what does it take to Prefetch where the industry is going? Be prepared to intercept. At that point is what you know, I, I refer to as you know, the, the speed factor in, in creating this mode, right? And the mode is just shed all things that hold you back and run as fast as you can. [00:02:53] Anush Elangovan: Uh, because the pace of innovation that is, uh, being seen in, in AI [00:03:00] industries is just. Amazing. Right? And it's like, it's transformational at at how you generate electricity. It's transformational as at how you build data centers. It's transformational at how you deploy compute, networking. It's transformational at what kind of use cases you, you know, uh, use AI for. [00:03:17] Anush Elangovan: Uh, and for that, you need to be prepared to, see what comes tomorrow and be prepared to run the race tomorrow. [00:03:23] Andrew Zigler: Yeah, it's a really great perspective because it highlights that it's not just like a checkpoint that you run through. I like how you called out, like it's not just hitting that benchmark or being the best in class at that moment, in that snapshot, it's about having a. The throughput and about having that dedication to the idea and continuing to deliver on it. [00:03:43] Andrew Zigler: It's not just crossing the threshold, but it's also being the engine. And that's what, that's what protects a business. That is the moat, because the moat is that innovation layer, the faster and more, uh, future forward. That you can work and think, [00:04:00] you know, the better. Uh, we, we talk a lot about like future forward work styles. [00:04:04] Andrew Zigler: Like what are the things I could be doing right now today that are gonna be like, way more useful tomorrow? Let, let's abandon those, workflows that are older and that kind of like, that translates into. An advantage when you work that way. You know, what kind of things have you learned working with, uh, like across all spectrums of people who would use ROCm, right? [00:04:23] Andrew Zigler: You have like the developers, but then you also have the enterprises and you have this large span of adoptees, right? So what is the, what does that look like that you learn? [00:04:32] Anush Elangovan: Yeah, so, so the way I look at it is there are gonna be pockets of different, uh, you know, cadences, right? Like, so people who are deploying in enterprises, for example, right? The validation and how long it takes for them to deploy an LLM that's secure. It's, with guardrails, et cetera, maybe longer. [00:04:52] Anush Elangovan: but you still have to go through the process and you have to be prepared to like, walk that walk to deploy an enterprises. That doesn't mean it's [00:05:00] not fast, that's as fast as you can do for that industry, right? And if you are deploying AI in healthcare, right, it's, it's got its own, uh, cycle. [00:05:07] Anush Elangovan: but in each one of these, you want to see how, like, go down to the essence of what is it that you actually have to do. And, you know, I, I, I like how you framed it. It's like it's, you shed your prior assumptions of how things are done, right. And, and you kind of build up from a, uh, first principles, uh, approach to say, this is how I could use AI to unlock, whatever I'm doing. [00:05:33] Anush Elangovan: And, and, some of it, you know, it's good to really step back and look at. Just question every part of it, right? Like right now you're getting chat GPT and, Gemini competing for like, math, olympiads and, and, uh, college, uh, reasoning, uh, tests. Right? And, and those are like that, that is amazing and increasingly like complex tasks that they're trying to do. [00:05:58] Anush Elangovan: But there may also be like. [00:06:00] More mundane things that AI could, could get applied to. Right? And, and so when we think about shedding old ways, you wanna shed it not just in like the tip of the spear. It's like, you know, I'm gonna see what's the frontier model. It's also, it could be something as simple as. [00:06:18] Anush Elangovan: How do you choose a, a movie, uh, you know, like a recommendation system, right? Or, or, uh, an automated, uh, flight, uh, rebooking system. So the moment, you know, your flight is late, uh, right now it's a notification, right? It's like, oh, you got a text message saying your flight's late. And I got that like three times this week. [00:06:38] Anush Elangovan: But anyway, uh, and, and, and, and, I was just like, okay, so if I were to rethink this. All this MCPs that we have that should be hooked up into an MCP that says, your flight's delayed. Here are your options. If you want, you know, these are the paid options. Yeah. Here are the free options. This will get you back into your you know, Toronto airport [00:07:00] tonight. [00:07:00] Anush Elangovan: Or if you stay, here's a hotel plus this, plus this, plus. It's just like, go ahead is all I should say. Versus now I'm like, okay, can someone, you know, can I call a travel agent? Can I do this? Can I go online and log into And you know, so we gotta fundamentally rethink even those like small, nuances of, things that we do that can be automated out and AI is really, really good at doing something like this, right? Maybe I just explained an AI startup idea right now. Somebody should just start that. [00:07:29] Andrew Zigler: I think you did. Yeah, you definitely did. Someone, one of our listeners is definitely going to lift that off of you. I, I, I, you know, I hate being on the receiving end of those. You feel a little helpless and then you have to like, follow the whole flow. So I know what you mean. Like I, I like how you called out that the build and this like. [00:07:45] Andrew Zigler: Where speed is your moat and the innovation layer is protecting you, is what makes you better than your competitors. How you scale that and you bring that to market. So by understanding the problems that you're solving, uh, throwing away those older assumptions, but also [00:08:00] recognizing that like. We're building every single day, new things and new ways of using stuff that we're still figuring out the implications of. [00:08:08] Andrew Zigler: And so when you have a lot of velocity and you're introducing a lot of new ideas, and maybe you have that workflow now that automatically rebook your flight off of your late flight text message, and uh, I know I would certainly use it, but you know, what kind of philosophies guide the way that y'all think about building this ecosystem to manage that stability while letting folks. [00:08:29] Andrew Zigler: Play with the speed and the assumptions and the airplane re bookings. [00:08:34] Anush Elangovan: so, so I think, you know, we need to peel one layer down, right? and the philosophy is, Hey, we, we just discovered electricity, right? And you know what we're gonna do? We are gonna make motors, uh, or dynamos, right? Like engines. Uh, sure. We don't know if it's gonna be a Ferrari that you're gonna make, or it's a a a a dump truck. [00:08:57] Anush Elangovan: That's good for doing this. But let's [00:09:00] let, which is also required, right? You need a dump truck. You need a garbage truck. And, [00:09:04] Andrew Zigler: Yeah. You need the [00:09:04] Anush Elangovan: course you need, uh, a Ferrari for a midlife crisis, right? So, [00:09:09] Andrew Zigler: precisely. [00:09:10] Anush Elangovan: But, but my, uh, point is what do we build next? And, uh, and this is what I meant by like, okay, let's, let's take those baby steps to build the. [00:09:20] Anush Elangovan: Infrastructure that's required that we know we'll have to use, right? So, so if I just discovered electricity, okay, great. Now one, how do I save this electricity and how do I use it? So there's battery technology, so you need to do something like that, right? Like so. But then you also want to make it into an actionable thing. [00:09:37] Anush Elangovan: You want to make it for like automobiles, or you wanna use it for, you know, powering, uh, entire cities. So it is that transformational. So, uh, AI is that transformational. So, if you distill down, it'll, it'll come down to how do we think about, what we can do with this this fundamental technology that, We may not be aware of what it [00:10:00] is gonna unlock next, but at least you know the next step is clear, right? It's like a dense fog, you know, it's gonna be like, it, it's the right path. You see the light, but it's kind of like out there and, and the steps you're taking are concrete and you're like, okay, this is good. [00:10:16] Anush Elangovan: I, this is better than where I was or where we were. So we are moving forward. So you can build with the. Intuition from what you see in the short term and a tactical view, but towards what you think the future is gonna be. [00:10:28] Andrew Zigler: Right. You almost like we're all in this like fog of war, right? And like you said, you're reaching out and you're trying to step through it. You could think of it too, as like you're in the dark and your hands are up in front of you and you know that. You're, you're not gonna run your face into a wall because your hands are out in front of you, but you're not gonna maybe do much better than that. [00:10:45] Andrew Zigler: So that's kind of like, I think the eco, the, the industry, the world that we find ourselves in, uh, and we all have to, then this becomes the power of an ecosystem, of a group of people working together to create that layer of, [00:11:00] uh, of establishing the [00:11:01] Anush Elangovan: exactly. And I, I, I just, instead of, you know, saying fog of war I describe it as like, you're in this. Beautiful valley with like a morning, uh, fog that's in. You can smell the flowers. You, you hear the birds. You are like, okay, it's, we are in like, uh, utopian paradise and yes, I just need to like, continue the walk, right? [00:11:24] Anush Elangovan: and then move forward with that, conviction that you're in the right spot. [00:11:27] Andrew Zigler: Yeah. So let's talk about that ecosystem world. This nice, I love how you describe it, this grassy side of a hill in the morning that's covered in some mist and maybe we can't see 30 feet in one direction, but it sure is a beautiful hill and it smells nice. And so we're all here. And why is, in that world, why is. [00:11:44] Andrew Zigler: You know, open source, their strategic advantage that y'all are going for in the AI hardware market. And, and then how does like ROCm turn that into wins for people within that ecosystem? [00:11:56] Anush Elangovan: you know, the, the way we look at it is this, is kind of like how I view [00:12:00] AI and the ecosystem, right? But, but it is for everyone to enjoy. Uh, and so we do want to make sure that. You know, it is, uh, beneficial for everyone. [00:12:09] Anush Elangovan: The ecosystem can come in and, and innovate. It's an open innovation engine. and uh, it is very different from, you know, having a walled garden with, Hey, only I know how to do this and I'm gonna do it and throw it over the fence and you can use it or keep walking, right? So we'd like to be good citizens that way, but also. [00:12:30] Anush Elangovan: Uh, it is self-fulfilling in a way, right? Like it, the, the pace at which we innovate with open source is unmatched. Like, you know, our serving engines are like VLLM and, and sg l. Those things, uh, those frameworks are like super, super aggressive in terms of how fast they come out with features and how fast they can you know, get performant models out. [00:12:52] Anush Elangovan: And that compared with what, uh, you'd get from, you know, the likes of like T-R-T-L-L-M or something is always lagging, right? Because you [00:13:00] just can't keep up with you know, 200 commits a week just on one particular model to get that model really performant [00:13:06] Andrew Zigler: And, and, and in that world where, you know, everyone can enjoy the winds of this, what kind of customer stories or innovation stories have really stood out to you and excite you about building and creating this place for developers? [00:13:19] Anush Elangovan: Yeah. So I think the parts that are super exciting for me are when when we get to see a customer that is first skeptical. Then they start a little like, okay, fine, we'll give you a chance. Uh, we do a simple, uh, POC and then they're like, huh, this seems to work. Yeah, we told you it works. [00:13:42] Anush Elangovan: You don't have to change one line of code. Really? Yes, no need to change one line of code. Okay, let's try a production workload. So then they try it. Oh, you're more performant than the competition. Yes. We're more performant than, than the competition. So how much does it cost? And we're like, oh, it's your TCO is better with, uh, [00:14:00] AMD. [00:14:00] Anush Elangovan: So again, they're like, wow, okay, good. So now how do we deploy at scale? And then we go deploy it at scale. And when they give a thumbs up on that and they say, this is good, right? That's when you know, you, you see it go full circle from like, oh, we, we've never heard about AMD to like actually deploy to tens of thousands of GPUs In the order of a few months, right? It, it, it really is fascinating to see and very exciting and invigorating to [00:14:28] Andrew Zigler: Yeah. At like a great exposure to a lot of interesting problems. And, and then people using the infrastructure, the, the technology available to solve those problems. Really specific problems by the way, that's often why they're bringing their data and AI to it, uh, is because it is really specific and important for them. [00:14:45] Andrew Zigler: And there's a, a lot I think that other engineering orgs can learn and even emulate from AMD's success and, and having this open source ecosystem and it causing this acceleration within. You [00:15:00] know, uh, customers and enterprises that use and adopt the tools and, and, and that creates an advantage. And that goes back to why we're talking and like the real thesis of our conversation today. [00:15:10] Andrew Zigler: So how do you think engineering leaders that are listening to this and obviously tapping into this great success AMD has from an open source flywheel, how do you think other, other folks building in the same space can foster that open, first, that open source oriented culture in order to, you know, accelerate their innovation goals? [00:15:29] Anush Elangovan: Yeah, that's a very good question. So the startup that um, was acquired by AMD we, we built, I mean, we started off doing iot stuff and you know, smart ring and all that, right? But in the, the end of like, uh, and not the end, the last six years of the company was building ML compilers. [00:15:47] Anush Elangovan: And ml, ML compilers are like super, uh, complicated, sophisticated, advanced algorithms, dah, dah, dah. but it was all open source, right? So our VCs were like, wait, what do you mean your core [00:16:00] IP is open source? And um, the speed is the moat applied even then, right? It was just like, yes, if you have an idea that. [00:16:08] Anush Elangovan: Because someone saw this idea that you are, they're gonna be able to catch up, then you probably have the wrong idea anyway. But if they are, you know, you execute and they're gonna catch up, that you should assume they're gonna catch up. Right? So you gotta move forward. So keeping it open source is super important. [00:16:25] Anush Elangovan: But also to your question on like, you know, the learnings from an AMD standpoint, right? If there are, hard problems, I'd say dig in and work through it, right? Like there's no way but through it, right? That should be the simple mentality. And more, uh, frequently than not. you'll see that you'll just make it through in a, in, in good form. [00:16:52] Anush Elangovan: But if you doubt it and you're like, oh, I don't know if I should commit, if I'm, I, you know, what should just commit to do the right thing [00:17:00] every step, right? Every step, and just keep taking one step in front of the other. And in no time you'll see that you'll be running. Right. And, and yes, the first few steps will be like, yeah, everyone's complaining about your software quality. [00:17:15] Anush Elangovan: Everyone's complaining about this and that, and it doesn't work. And, and a few steps in, you know, you get, you get the hang of all the complaints that are coming in. You get the feedback loop. You're like, okay, what, what are you prioritizing again? One step in front of the other, right? You just keep knocking that out and then you get to a point where you're, it just becomes second nature, right? To do the, to do the right thing. And, and then yes, if someone gives you two options, you'll be like, fine. This is, uh, you know, there's always the resource trade off. There's always a human capital trade off, but what's the right thing to do? of course, I, I'm pragmatic about what we choose, but, but if the right thing for your long-term success is dig in, go first, principles, make it [00:18:00] happen. [00:18:00] Anush Elangovan: Well. Then just go for that. There's, there is no shortcut to [00:18:04] Andrew Zigler: acknowledging, you know, how it aligns with your mission, your core company goals, and what you're looking to achieve. And, and I, I love how you rightfully called out that in the open source world and you know, you have your technology that you've built, what you think is your moat upon, right? [00:18:22] Andrew Zigler: It's your code and, and to open source that, or to just make it where anyone could peer in is, you know. Scary in one regard, but two, it just kind of feels like you're handing away your throne room in some kind of sense, a very direct feeling sense. But the ultimately, you were really right to call out, and this is something I think about all the time, that the real power there is still the speed This the speed. [00:18:42] Andrew Zigler: That was the moat at the beginning of our conversation. It's the speed in combination with your. Very specific domain understanding of what you're building and what you're creating, and your new role as the steward of that world and how people plug into it, which [00:19:00] has frankly, a lot more influence and power than lording over a closed. [00:19:04] Andrew Zigler: You know, repository or an ecosystem, and like you said, like throwing things over the wall. Sure. There, there might be people always on the other side of that wall, but you're not gonna have a great connection with them. You're not gonna be able to really clearly understand them. I, I like your metaphor of the side of the field of the mountain a lot more. [00:19:23] Andrew Zigler: But, but in the, in this world, you know, where. That speed is, is the power and, and open source is just one way that you can harness that speed to get really far ahead and to innovate. , There's other parts of this equation that you can be experimenting with too, and I'd love to pick your brain about them as a software leader and, and, and one of them is about looking forward and kind of understanding that future that we're all building towards and beyond today's models and hardware. [00:19:48] Andrew Zigler: You know, what do you see as the next major bottleneck or opportunity in the AI compute space? As, as you know, enterprises and folks start to get a little more mature about what's available to [00:20:00] them. [00:20:00] Anush Elangovan: Yeah, I think, the bottleneck and opportunity is, uh, what I'd call, call walking the last mile of ai. Right. Uh, and like I I, I gave you an example, uh, previously, but, but it's similar to that. It's like there are cases where Humans have so many, uh, things to do in your day. You know, like the, if we sit down and actually had a customer focus like, okay, these customers lives, I'm gonna save four hours of this customer's life. And if you actually sit down and look at all of that, it'll be. Easily automatable, easily you know, uh, applicable, uh, for ai, right? [00:20:39] Anush Elangovan: Like, but then making it happen is gonna take a little bit, right? It's like maybe it's, uh, paying your utility bill, right? Or something like that, right? Or, or, your healthcare explanation of benefits. Uh, like, I'm sure you get an explanation of benefits, and I'm like, I, I don't even know what that thing is. [00:20:55] Anush Elangovan: It's just like EOB and like. [00:20:57] Andrew Zigler: it's a big, a big old PDF. Yeah, [00:21:00] exactly. [00:21:01] Anush Elangovan: Like, like, I'm like great straight to the, uh, shredder, right? And but that could be, you know, automated with the ai, right? It, it, it'd be like, Hey, the summary of this thing is you went and visited this day. Everything is okay. Everything is paid for, so don't worry, it's not a bill. [00:21:17] Anush Elangovan: That again, the same, uh, thing, but the sense of what that information overload is could be. Digested by ai, uh, accumulated over time and retrieved when you need it. Like, I don't, I actually don't even need to know this EOB right now, unless of course, whenever I need to know it, that maybe, you know, like for some benefits I need to figure out what do, what did I do over the past year and how do I apply it? Source:

Mike

15,248 görüntüleme • 9 ay önce

$Cloud capital versus AI - What DeepSeek’s spectacular success means for technofeudalism & the New Cold War DeepSeek, a Chinese artificial intelligence (AI) company, this week changed the global AI landscape, not to mention caused $1 trillion losses in the New York stock exchange and the NASDAC. In the process, it demonstrated the difference between cloud capital, which drives technofeudalism onward and upward, and AI-services, which were always a bubble waiting to burst. What remains to be seen is DeepSeek's impact on the New Cold War between the US and China which, from its beginning, was motivated by the clash between US and Chinese cloud capital. DeepSeek is China's response to OpenAI's ChatGPT. Its models perform as efficiently as their US counterparts. The difference is that DeepSeek is offered for free, making money only by selling services to developers - not to the public – at a fraction of the price OpenAI charges! The gist of DeepSeek’s arrival on the AI scene is a sudden transition from proprietary to open source technology. It is, therefore, no great wonder that, the moment DeekSeek became the most downloaded app on the Apple Store, it pulverised the market capitalisation of the, hitherto overinflated, US Big Tech companies. But how did this happen? How is a private commodity suddenly being offered for free? And does this mean that technofeudalism is in trouble? To begin with it is important to note AI was never a proprietary technology in itself. The underlying code has always been open source. What made AI quasi-private was the way these models were trained using huge amounts of privatised (that is stolen from us) data. A leaked Google memo in 2017, that was widely discussed in the industry at the time, but also widely refuted, explained: "If an open source LLM trained for a few million dollars outperforms the effectiveness of proprietary models... There will be no firewall to safeguard OpenAI either." DeepSeek pierced the US AI companies’ bubble by decommodifying the results of the model’s training, shifting them from behind a paywall to the public arena. Within days, developers around the world started building their own models on top of DeepSeek's. This is was the nightmare for US Big Tech's AI service providers who offered the results of prompts as a commodity, in the form of subscriptions. DeepSeek-type applications can now produce high-quality translations for free and, in so doing, undermine companies specialising in, for example, translation services, such as Germany's Deepl. In the broader scheme of things, this means that the morsels of cloud capital that Europe owns has lost its market value. Nevertheless, and this is a huge nevertheless, it is only AI-as-a-commodity that has lost its (grossly exaggerated) value. In sharp contrast, cloud capital utilised not as a commodity producing piece of tech but as produced means of behavioural modification is not at all threatened by companies like DeepSeek. And since technofeudalism is powered by cloud capital working that way, rather than commodity-like AI services of the ChaptGPT type, our technofeudal order is not threatened by competitors such as DeepSeek. To help understand the difference between cloud capital and AI-based commodified services it helps to compare and contrast Alexa and ChatGPT. Alexa is not offering you a commodified service. It is your free pretend-slave. Unlike ChatGPT you do not pay a subscription to Amazon for the right to order Alexa to order you milk or to switch off your lights. Instead, you train Alexa to train you to train it to know you so that it wins your trust with good recommendations so that it can modify your behaviour – ‘encourage’ you to buy a commodity from with Bezos retaining up to 40% of the price you pay (as a cloud rent). In short, the work that Alexa performs for you is not a commodity, unlike ChatGPT which works to sell you a commodity. In other words, ChatGPT is subject to market competition, to the likes of DeepSeek, but Alexa is not. This is why OpenAI, ChatGPT’s maker, is seriously damaged by the emergence of DeepSeek but Amazon is not. Thus, my basic point: Cloud capital is in a league of its own, beyond market competition from DeepSeek-like upstarts, because its power lies in its capacity to modify our behaviour and remove us from any market (e.g., to shift us from real markets to cloud fiefs like Amazon and Alibaba). In conclusion, cloud capital’s capacity to drive technofeudalism is not challenged by companies like DeepSeek. Only companies like OpenAI, which invested so much and so foolishly in providing a commodified service, stand to lose enormously. Yet another sign that capitalism is dead at the hand of cloud capital while technofeudalism is going from strength to strength and, as it does so, fuels even further the New Cold War between the US and China which in my book, Technofeudalism, I have explained away as the clash of the two huge concentrations of cloud capital: the American dollar-denominated super cloudalist power and the Chinese yuan-denominated one. Speaking of this New Cold War, which I have argued is mostly fuelled by the clash between American and Chinese cloud capital, I wonder what impact DeepSeek’s success will have on the US government. Silicon Valley and Washington DC had convinced themselves that America had a huge AI lead over China. Now, a tiny Chinese company has destroyed that confidence by producing on a shoestring better AI tech than Sillicon Valley had imagined possible. I can almost hear the whirring inside the heads of people in power on both America’s East and West Coast thinking that if the Chinese can do this out of the blue, what else can they do tomorrow? It is reminiscent of the Sputnik moment, isn’t it? It will be interesting to see how Trump reacts to this threat to companies American AI companies, especially since Elon Musk understands, and has spoken out against, the folly of commodifying AI services rather than going full on technofeudal. These are interesting times, in the traditional Chinese sense of the phrase.$

Cloud capital versus AI - What DeepSeek’s spectacular success means for technofeudalism & the New Cold War DeepSeek, a Chinese artificial intelligence (AI) company, this week changed the global AI landscape, not to mention caused $1 trillion losses in the New York stock exchange and the NASDAC. In the process, it demonstrated the difference between cloud capital, which drives technofeudalism onward and upward, and AI-services, which were always a bubble waiting to burst. What remains to be seen is DeepSeek's impact on the New Cold War between the US and China which, from its beginning, was motivated by the clash between US and Chinese cloud capital. DeepSeek is China's response to OpenAI's ChatGPT. Its models perform as efficiently as their US counterparts. The difference is that DeepSeek is offered for free, making money only by selling services to developers - not to the public – at a fraction of the price OpenAI charges! The gist of DeepSeek’s arrival on the AI scene is a sudden transition from proprietary to open source technology. It is, therefore, no great wonder that, the moment DeekSeek became the most downloaded app on the Apple Store, it pulverised the market capitalisation of the, hitherto overinflated, US Big Tech companies. But how did this happen? How is a private commodity suddenly being offered for free? And does this mean that technofeudalism is in trouble? To begin with it is important to note AI was never a proprietary technology in itself. The underlying code has always been open source. What made AI quasi-private was the way these models were trained using huge amounts of privatised (that is stolen from us) data. A leaked Google memo in 2017, that was widely discussed in the industry at the time, but also widely refuted, explained: "If an open source LLM trained for a few million dollars outperforms the effectiveness of proprietary models... There will be no firewall to safeguard OpenAI either." DeepSeek pierced the US AI companies’ bubble by decommodifying the results of the model’s training, shifting them from behind a paywall to the public arena. Within days, developers around the world started building their own models on top of DeepSeek's. This is was the nightmare for US Big Tech's AI service providers who offered the results of prompts as a commodity, in the form of subscriptions. DeepSeek-type applications can now produce high-quality translations for free and, in so doing, undermine companies specialising in, for example, translation services, such as Germany's Deepl. In the broader scheme of things, this means that the morsels of cloud capital that Europe owns has lost its market value. Nevertheless, and this is a huge nevertheless, it is only AI-as-a-commodity that has lost its (grossly exaggerated) value. In sharp contrast, cloud capital utilised not as a commodity producing piece of tech but as produced means of behavioural modification is not at all threatened by companies like DeepSeek. And since technofeudalism is powered by cloud capital working that way, rather than commodity-like AI services of the ChaptGPT type, our technofeudal order is not threatened by competitors such as DeepSeek. To help understand the difference between cloud capital and AI-based commodified services it helps to compare and contrast Alexa and ChatGPT. Alexa is not offering you a commodified service. It is your free pretend-slave. Unlike ChatGPT you do not pay a subscription to Amazon for the right to order Alexa to order you milk or to switch off your lights. Instead, you train Alexa to train you to train it to know you so that it wins your trust with good recommendations so that it can modify your behaviour – ‘encourage’ you to buy a commodity from with Bezos retaining up to 40% of the price you pay (as a cloud rent). In short, the work that Alexa performs for you is not a commodity, unlike ChatGPT which works to sell you a commodity. In other words, ChatGPT is subject to market competition, to the likes of DeepSeek, but Alexa is not. This is why OpenAI, ChatGPT’s maker, is seriously damaged by the emergence of DeepSeek but Amazon is not. Thus, my basic point: Cloud capital is in a league of its own, beyond market competition from DeepSeek-like upstarts, because its power lies in its capacity to modify our behaviour and remove us from any market (e.g., to shift us from real markets to cloud fiefs like Amazon and Alibaba). In conclusion, cloud capital’s capacity to drive technofeudalism is not challenged by companies like DeepSeek. Only companies like OpenAI, which invested so much and so foolishly in providing a commodified service, stand to lose enormously. Yet another sign that capitalism is dead at the hand of cloud capital while technofeudalism is going from strength to strength and, as it does so, fuels even further the New Cold War between the US and China which in my book, Technofeudalism, I have explained away as the clash of the two huge concentrations of cloud capital: the American dollar-denominated super cloudalist power and the Chinese yuan-denominated one. Speaking of this New Cold War, which I have argued is mostly fuelled by the clash between American and Chinese cloud capital, I wonder what impact DeepSeek’s success will have on the US government. Silicon Valley and Washington DC had convinced themselves that America had a huge AI lead over China. Now, a tiny Chinese company has destroyed that confidence by producing on a shoestring better AI tech than Sillicon Valley had imagined possible. I can almost hear the whirring inside the heads of people in power on both America’s East and West Coast thinking that if the Chinese can do this out of the blue, what else can they do tomorrow? It is reminiscent of the Sputnik moment, isn’t it? It will be interesting to see how Trump reacts to this threat to companies American AI companies, especially since Elon Musk understands, and has spoken out against, the folly of commodifying AI services rather than going full on technofeudal. These are interesting times, in the traditional Chinese sense of the phrase.

Yanis Varoufakis

96,399 görüntüleme • 1 yıl önce

Has been a while since I've given an update so here's a breakdown of where Sappy is at right now and what we're focusing on going into this year. Pre-amble: With altcoins & NFTs the market is definitely not the same as it was before. I think this is obvious to everyone but I've noticed there are still japanese soldiers that are convinced old tricks and mechanics work. They don't. Liquidity is thin; people want to bid assets that feel like "real companies" not vacuous memecoins. There's still room for memecoins, social currencies, and "utility tokens" (I would say without these functions, tokens are hard to justify versus equities). I'm not part of the camp that thinks there will never be hyperspeculation in crypto again, because there will be; we all love ponzis and PvPing each other onchain. Just not with solved games -- people need something new and fresh. So the overarching plan is to continue building for users, sustainable revenues that aren't tied to directly to crypto, and doubling down on the areas that we've already found PMF / Brand Market Fit. Then leaning into crypto during cyclical periods where liquidity is sloshing around at an accelerated rate. Where we've found early PMF / what we're leaning into: Roblox: we're going to continue to go hard and accelerate here. It's our main objective to ship more seal/brainrot focused games across most genres to cast as wide of a net as we can for the brand, and to also iterate and see what works and stays sticky. Our initial incursion into Roblox was very successful peaking at 2M+ MAU and still sustaining a large portion of that player base... for all of its success, that was a relatively amateur first attempt; we've been setting up better AI pipelines for Roblox development that makes it reasonable to ship many more games and 10x those player counts in totality. It's my belief that Roblox is the sandbox whose audience will be the most valuable on the internet once they are grown up. That intense feeling you get when you see a TikTok referencing an old game you enjoyed on the PS2 or the Gamecube, or when you see a Pokemon card is the exact same feeling the youth of today will get when reminiscing on the things they enjoyed engaging with when they were younger. Fortnite and Roblox are functional equivalents to the old school consoles and exactly where that is taking place. Which is why as much as I care about scaling revenues through Roblox, the long term brand equity gained purely through being popular on the platform is totally invaluable. It also can heavily convert to merchandise sales today if all touchpoints for the brand are dialed in (which is why brands get overcharged so much by Roblox dev shops for the same ROI that only cost us a few thousand $). We have the playbook, it's just about iterating new concepts and then aggressively scaling. Brand Expansion & Merchandising: I've started to create a content pipeline that is easily repeatable, cost efficient (costs next to nothing through either AI or smart reusable concepts), while still being very tasteful and meeting our quality standards for the brand. We are mostly focusing here on reaching people where they're at through nostalgic/emotional content, or just being visually stimulating through carefully curated aesthetics. Content that isn't superficial and touches people in a memorable way. I've attached some examples to the post so you can see what I mean rather than just read it. I don't think it's long until larger brands start doing this at scale, but it's always good to be ahead of the curve and most importantly winning on taste -- knowing what will resonate with people and what won't has always been our edge. The purpose for these accounts is not only to rack up attention but also to begin converting those into sales of both of physicals (plushies & gacha collectibles) and digital avenues like our games, and any other apps we produce. Because they're offshoot accounts it's also a lot easier to be aggressive/experimental with said conversion strategies. Sappy Studio: I'm wrapping everything like Omnia, and everything else into this category because they're all tangentially related. Beginning with Omnia, our current focus is gearing up for Season 0 which involves players competing in the ranked ladder for a prize pool that has rewards through Monad Momentum as well as a player-funded prize pool. This season will be fairly simple with us mostly logging retention, deck building habits, as well as qualitatively observing how aggressively players push the combat system. Deeper monetization wont exist yet outside of the player buy-in (to be eligible for P2E rewards). Beyond that our overarching principle this year is to focus heavily on risk-to-earn mechanics where a portion of that excess value is circular i.e. revenues flow back to prize pools or other parts of the economy, treating the game almost like a protocol where the objective is to amass TVL or player liquidity. Social is also a big focus, and that means implementing the Open World hub which from an infrastructure perspective has already been built out and tested by all of you previously. Right now we are scaffolding the environment in 3D and working through how that hub should look and feel, so players are excited to hang out & idle together while they're queuing. For sappydotlol, what I'm about to say is still early days from a design perspective so a lot can change, but I'm pushing the site in the direction of being a virtual game console. An intersection between Nintendo & Myspace where users can play, trade, and socially interact in a way that's deeply personalised; a breathe of fresh air from the hostility of the current internet. If you go back to my thesis on Roblox above and the game console references, you can kind of see how this will all sequentially tie together. In essence, the strategy is to acquire a critical mass of players through traditional platforms like Roblox, and use that attention and trust to provide an onboarding funnel for web2 users into our own sandbox filled with a mixture of our own browser-based experiences as well as an aggregation of others. The aim is to make the platform a breath of fresh air & bunker from the enshittified platforms like TikTok/IG/X where users are actually served in ways that delight rather than agitate, and where self-expression is incentivised. Closing: As always everything here is subject to change but I've never felt more conviction in our direction until now; I know exactly what we need to do and how, with everything aligning with our team's strengths. Very excited and grinding through things to the point where I'm getting headaches and can't sleep from being hyperfocused for long periods of time lol. There probably has never been a better time to join the ecosystem from a price to fuck around and find out perspective.

wab.eth

18,296 görüntüleme • 8 ay önce

$NVDA $GFS NVIDIA’s reported agreement to acquire Groq for $20B in cash (per CNBC, amplified via Reuters and other wire coverage) represents a materially different strategic posture than NVIDIA’s prior M&A pattern, given both the headline size (largest reported NVIDIA acquisition to date) and the unusual carve-out that Groq’s early-stage cloud business would not be included. Public reporting indicates the information originated from Alex Davis, CEO of Disruptive (lead investor in Groq’s latest financing), and that neither NVIDIA nor Groq had issued an immediate confirmation at the time of publication. The same reporting frames the transaction as coming together quickly, only months after Groq raised $750M at a ~$6.9B valuation, and highlights Groq’s positioning as a high-performance inference chip vendor founded by ex-Google TPU engineers. Groq is best understood as a vertically integrated inference acceleration company whose core asset is an application-specific processor optimized for deterministic, low-latency execution of transformer-style workloads, paired with a compiler-led software stack and a distribution layer (GroqCloud) designed to reduce developer friction via OpenAI-compatible APIs and integrations. Groq brands its architecture as a Language Processing Unit (LPU) and consistently emphasizes that the design target is inference, not training. The company’s own architecture description centers on 1-core execution, large on-chip SRAM used as primary storage (explicitly not cache), a custom compiler that statically schedules compute and communication, and direct chip-to-chip connectivity intended to coordinate multi-chip execution without relying on conventional caching hierarchies or dynamic runtime scheduling. The technical premise is a deliberate inversion of the conventional GPU approach. GPUs deliver throughput via massively parallel, multi-core execution with dynamic scheduling, complex memory hierarchies, and heavy reliance on off-chip HBM bandwidth and sophisticated runtime/kernel optimization. Groq instead argues that inference bottlenecks are driven by latency variance (tail latency), synchronization overhead, and memory access unpredictability inherent in dynamically scheduled, cache-heavy architectures, particularly when workloads are latency sensitive and batch sizes cannot be inflated. Groq’s solution is to move “control” into the compiler: the full execution graph and inter-chip communication schedule are computed ahead of time down to clock-cycle granularity, with deterministic execution designed to reduce run-to-run variance. In Groq’s framing, the removal of caches, reorder buffers, speculative execution overhead, and other sources of contention enables predictable latency and high utilization without per-model kernel engineering typical of GPU tuning cycles. A critical nuance is that Groq’s determinism is not merely a software claim; it is tightly coupled to architectural constraints and system design choices that trade flexibility for predictability. Third-party technical commentary indicates Groq’s chip uses a fully deterministic VLIW-style approach with minimal buffering, no external memory, and heavy dependence on sharding models across many chips because on-chip SRAM capacity is limited. SemiAnalysis describes a ~725 mm^2 die on GlobalFoundries 14nm with ~230MB of SRAM and notes that “no useful models” fit on a single chip, forcing multi-chip partitioning for modern LLMs and driving a system-level design where networking and compilation are first-class scheduling problems rather than ancillary infrastructure. This is consistent with Groq’s own messaging that tensor parallelism across chips is a primary design goal, enabled by large on-chip SRAM and compile-time coordination of compute plus interconnect. The on-chip SRAM emphasis is central to Groq’s latency story and also its most constraining trade-off. Groq claims on-chip SRAM bandwidth “upwards of 80 TB/s” and contrasts that with off-chip HBM bandwidth “about 8 TB/s,” asserting a potential 10x advantage from bandwidth plus reduced trips across chip-to-memory boundaries. While these comparisons are marketing-oriented and depend on workload specifics, the architectural implication is clear: Groq prioritizes ultra-fast local weight/activation access and then scales capacity by adding chips, not by attaching large off-chip memory pools. This design can reduce latency for sequential inference layers and minimize unpredictable stalls, but it pushes complexity into partitioning strategy, interconnect topology, and compiler scheduling, and it increases the number of chips needed for very large parameter counts and large KV-cache footprints. Groq also highlights numeric formats and compiler-driven precision management as a performance lever. In its 2025 technical blog, Groq describes “TruePoint numerics,” including 100-bit intermediate accumulation and selective quantization choices (FP32 for attention-sensitive operations, block floating point for MoE weights, FP8 storage in error-tolerant layers), and claims 2-4x speedups versus BF16 without measurable accuracy degradation on benchmarks such as MMLU and HumanEval. Even if the absolute uplift is workload dependent, the strategic point is that Groq is pursuing performance via end-to-end co-design: precision policy is not just hardware capability (FP8/BF16) but compiler-enforced mapping of precision to error sensitivity, which can matter materially for inference cost-per-token if it reduces memory traffic and boosts throughput without forcing aggressive, accuracy-damaging quantization. Independent performance datapoints indicate Groq has been credible on latency-oriented inference speed, at least for certain regimes. EE Times reported in 2023 that Groq demonstrated Llama-2 70B inference at ~240 tokens/s per user on a cloud-based dev system described as 10 racks and 64 chips, using the company’s 1st-gen silicon introduced several years earlier. Separate Groq commentary around independent benchmarking cites results showing ~241 tokens/s throughput and ~0.8s time to receive 100 output tokens for a Llama-2 70B API configuration, positioning the platform as a step-change in “available speed” for certain interactive use cases. These figures do not settle total cost-of-ownership versus GPUs or hyperscaler ASICs, but they establish that Groq’s system-level architecture can deliver strong single-user throughput and latency on large models when properly partitioned and scheduled. GroqCloud is the commercial wrapper that packages this hardware/software stack as “tokens-as-a-service,” aiming to make Groq adoption feel like switching API endpoints rather than adopting new silicon. Groq’s documentation states its API is designed to be “mostly compatible” with OpenAI client libraries, and its pricing page provides model-specific token rates, published speeds (tokens/s), prompt caching discounts, and batch processing discounts. For example, pricing lists inputs as low as $0.05 per 1M tokens and outputs as low as $0.08 per 1M tokens for certain smaller LLM configurations, with higher prices for larger models and long-context or MoE variants; it also advertises prompt caching with a 50% discount on cached input tokens for certain models and a batch API offering 50% lower cost for asynchronous processing windows. These mechanics are economically important because they demonstrate Groq’s go-to-market is not simply “sell chips,” but “sell predictable unit economics per token,” with tooling (batch, caching) that directly targets inference cost drivers (reused prompts, throughput smoothing, and asynchronous workloads). The cloud footprint and distribution partnerships indicate Groq has been building an inference-native “edge within the cloud” strategy rather than competing head-on with hyperscalers on breadth of services. A 2025 Groq newsroom release describes a European deployment in Helsinki with Equinix, positioned as latency reduction and data governance for European customers, and explicitly references Equinix Fabric enabling private connectivity to GroqCloud over public, private, or sovereign infrastructure. The same release enumerates additional capacity in the U.S. (Equinix, DataBank), Canada (Bell Canada), and Saudi Arabia (HUMAIN), and states these sites collectively served more than 20M tokens/s across Groq’s global network at that time. That supply-side metric matters because it provides a directional sense that Groq is scaling capacity as a network, not merely as a chip vendor. Customer disclosure is inherently limited because Groq is private and many enterprise deployments are not public, but Groq’s marketing materials and partnerships provide signals about demand vectors. The company’s public website displays logos of large consumer and enterprise brands (e.g., Dropbox, Vercel, Chevron, Volkswagen, Canva, Robinhood, Riot Games, Workday, Ramp) and includes a published customer quote claiming a 7.41x chat speed increase and an 89% cost reduction after moving to GroqCloud, followed by a tripling of token consumption. While marketing claims should be treated as case-specific and not generalized, they indicate that Groq is targeting both AI-native developers (who measure success by latency and cost-per-token) and enterprise buyers (who care about predictable performance and governance). Supplier and dependency mapping for Groq spans 3 layers: silicon production, system integration, and cloud infrastructure. On silicon, third-party analysis indicates GlobalFoundries 14nm for the 1st-gen Groq chip, implying a supply chain less constrained by the most capacity-tight leading-edge nodes and advanced packaging bottlenecks that dominate high-end GPU supply (HBM stacks, CoWoS-type packaging constraints). If accurate, this is strategically meaningful because it suggests Groq capacity expansion could be gated more by conventional wafer supply, board assembly, and data center power than by the same HBM/advanced packaging scarcity that has constrained top-tier GPU ramp cycles. On systems and cloud, Groq’s own releases identify colocation and connectivity partners (Equinix, DataBank, Bell Canada) and a Middle East partner (HUMAIN), implying dependencies on data center real estate, power availability, and network connectivity, alongside procurement of standard server components, NICs/switching, racks, and cooling infrastructure. The Groq design narrative also emphasizes air cooling and reduced need for complex power/cooling infrastructure, which—if realized in deployments—can widen the set of feasible hosting locations and lower deployment friction relative to liquid-cooled, very high power density GPU racks. Against that backdrop, the strategic rationale for NVIDIA acquiring Groq can be framed as a set of overlapping objectives: inference silicon optionality, architectural hedging, competitive defense, and supply chain diversification, with the carve-out of GroqCloud signaling a preference to avoid direct cloud competition and to focus on IP and product portfolio control rather than operating a capital-intensive token-serving business. The deal, if confirmed, would occur at a valuation step-up of ~190% versus Groq’s reported ~$6.9B private valuation in the September $750M round, reinforcing that any acquisition logic would be predominantly strategic rather than a conventional financial multiple arbitrage. The most compelling strategic driver is inference. Training has historically been the center of gravity for cutting-edge GPU demand, but inference volume is structurally larger and more distributed as deployments scale, with economics dominated by cost-per-token, latency guarantees, and utilization under spiky demand. Inference workloads also create a strategic vulnerability for NVIDIA: hyperscalers and large platforms can justify bespoke ASICs (TPU, Trainium/Inferentia, Maia-class efforts) because inference is stable, repeatable, and can amortize software investment at massive scale. Groq’s core proposition—deterministic, compiler-scheduled inference with predictable latency—aligns directly with the segment where GPU generality is least valued and where “good enough” programmability plus superior unit economics can win share. Acquiring Groq would allow NVIDIA to own a credible inference-native architecture rather than relying solely on GPUs and software optimization to defend that segment. Competitive defense logic is also plausible. Groq occupies a specific competitive wedge: low-latency, high-throughput interactive inference, delivered via a simple API abstraction that reduces switching cost. That wedge directly pressures GPU inference margins in the long run because it makes inference price/performance comparisons more transparent at the token level, and it targets a developer persona that historically defaulted to CUDA-first ecosystems. Even if NVIDIA’s current-generation systems can achieve very high tokens/s per user with extensive optimization, the strategic risk is that competing architectures normalize the idea that inference is best served by special-purpose silicon with a simpler programming model, weakening CUDA lock-in at the application layer. NVIDIA has actively demonstrated that Blackwell-era systems can exceed 1,000 tokens/s per user in benchmarked configurations, but that performance leadership does not automatically translate to lowest cost-per-token across the full range of batch sizes, latency targets, and deployment environments. Groq’s existence as a credible alternative architecture forces NVIDIA to keep defending inference economics rather than only raw performance leadership. The “technology acquisition” rationale is unusually strong in this specific case because Groq’s differentiator is not a single block of silicon IP but an end-to-end methodology: compiler-led static scheduling, deterministic networking, and a system architecture designed around tensor-parallel inference rather than throughput-maximizing batch inference. NVIDIA’s stack is already compiler-heavy (TensorRT, Triton, CUDA graphs, kernel fusion, speculative decoding techniques), but GPUs remain dynamically scheduled devices with complex memory hierarchies and stochastic latency behaviors under contention. Groq’s approach provides an alternate design point: treating the entire inference execution (compute plus communication) as a statically schedulable program. In principle, that IP could be valuable even if Groq silicon itself is not adopted at massive scale, because it can inform how NVIDIA builds future inference-optimized products, compilers, and networking fabrics, especially as distributed inference with large models makes communication a first-order performance determinant. Supply chain diversification is a non-obvious but potentially important driver. If Groq’s mainstream product generation is truly based on a mature process node and avoids HBM, then the scaling constraints look different than those of state-of-the-art GPUs. NVIDIA’s ability to meet incremental demand has been tightly coupled to advanced packaging and HBM supply, and those constraints can remain binding even when wafer supply is available. An inference ASIC architecture that relies primarily on on-chip SRAM and scales by adding chips—while not costless—could reduce dependence on HBM availability and advanced packaging capacity, enabling NVIDIA to ship “inference capacity” in higher absolute volumes or into geographies and customer segments where the highest-end GPUs are economically or logistically difficult to deploy. This could be particularly relevant for latency-sensitive inference deployed in regional colocation footprints rather than centralized hyperscale campuses. The carve-out of GroqCloud, if accurate, is itself a strategic signal about NVIDIA’s priorities. Operating a token-serving cloud at scale is capital intensive, structurally lower margin than silicon IP rents, and creates channel conflict with hyperscalers and CSP partners who are core NVIDIA customers. NVIDIA has generally positioned its cloud offerings through partnerships rather than as a direct hyperscale competitor. Excluding GroqCloud would preserve neutrality with CSPs and avoid inheriting multi-region data residency obligations and partner contracts, while still allowing NVIDIA to acquire Groq’s silicon, compiler technology, and engineering talent. At the same time, excluding GroqCloud would also mean NVIDIA would not automatically acquire the commercial proof-point of Groq’s unit economics or the customer contracts that validate product-market fit at scale, increasing the importance of diligence on whether Groq’s cloud pricing is structurally profitable or partially subsidized by fundraising. There is also a “preemptive acquisition” angle. The reporting identifies recent investors in Groq’s latest round including large financial institutions and strategic/industry players. In that context, Groq represents an asset that could plausibly have been acquired by a competitor (AMD/Intel) or by a hyperscaler seeking to accelerate inference independence. NVIDIA acquiring Groq could be a defensive move to prevent a credible inference-native architecture from being weaponized by a rival with deep distribution. Even if GroqCloud is carved out, controlling the silicon roadmap and compiler IP would meaningfully constrain Groq’s ability to evolve into a standalone competitor, unless the carved-out entity retains long-term rights to the hardware and software stack. However, the strategic case is not one-sided; there are meaningful risks and potential contradictions that would need to be reconciled for the transaction to be value-accretive on a multi-year horizon. 1st, Groq’s architecture appears to rely on scaling out chip count to achieve capacity, which introduces system cost, networking complexity, and physical footprint considerations. The absence of external memory and limited on-chip SRAM implies very large models require substantial chip parallelism, and the economics then depend heavily on chip cost, yield, power efficiency, and interconnect overhead. SemiAnalysis explicitly frames Groq as trading space for time and raises questions about token economics and whether publicly advertised pricing reflects fully loaded costs or market share capture. 2nd, integration risk is non-trivial. Groq’s compiler-led deterministic model is philosophically and practically different from CUDA’s dominant programming and execution model. A poorly executed integration could create internal product confusion, dilute engineering focus, or alienate developers if the combined stack fragments. 3rd, there is cannibalization risk. If Groq-class inference silicon undercuts GPU inference economics, NVIDIA could face internal margin trade-offs, even if the goal is to defend share against hyperscaler ASICs. Cannibalization can still be rational if it prevents larger share loss, but it would require crisp portfolio segmentation and go-to-market discipline. The presence of NVIDIA’s own rapidly improving inference performance complicates the “need” for Groq but does not eliminate the “option value.” NVIDIA has demonstrated benchmark-leading tokens/s per user on Blackwell-based systems, suggesting that raw interactive throughput is not necessarily the limiting factor for NVIDIA’s product line. The more enduring strategic question is unit economics and architectural control: whether future inference demand is better monetized through general-purpose GPUs plus software optimization, or whether a bifurcated product portfolio (training GPUs plus inference-native ASICs) becomes necessary to defend total AI compute wallet share as hyperscaler ASIC penetration increases. Acquiring Groq could be a decisive move to ensure NVIDIA participates in both regimes rather than betting exclusively on GPUs to win inference forever. What is “special” about Groq’s technology relative to a typical accelerator roadmap is the tight coupling of determinism, compilation, and networking into a single scheduling problem. The LPU narrative emphasizes deterministic compute and networking, static scheduling, and direct chip-to-chip coordination that allows “hundreds” (more precisely, 100s) of chips to behave like a single scheduled resource. The architecture also explicitly targets tensor-parallel, latency-optimized distribution rather than pure data-parallel throughput scaling, which matters for real-time applications where a single response must arrive quickly rather than many requests being processed in bulk. The implication is that Groq is optimized for the time-to-first-token and steady token streaming behavior that defines user experience in interactive LLMs, and it attempts to achieve that without relying on large batch sizes that can degrade latency. From a portfolio manager’s perspective, the most important interpretation is that an NVIDIA-Groq combination would likely be less about “NVIDIA needs more inference speed” and more about controlling the architectural trajectory of inference acceleration and removing a fast-improving, developer-friendly competitor from the market. The carve-out of GroqCloud would reinforce that the transaction is aimed at IP, talent, and product optionality, not acquiring a cloud revenue stream. The valuation step-up implied by $20B versus $6.9B would therefore be justified only if the acquired assets materially reduce long-term competitive risk (hyperscaler ASIC displacement, inference margin compression) or enable new monetization vectors (inference ASIC product line, supply chain de-bottlenecking, improved software determinism) that would be difficult to achieve on a comparable timeline via internal R&D.

$NVDA $GFS NVIDIA’s reported agreement to acquire Groq for $20B in cash (per CNBC, amplified via Reuters and other wire coverage) represents a materially different strategic posture than NVIDIA’s prior M&A pattern, given both the headline size (largest reported NVIDIA acquisition to date) and the unusual carve-out that Groq’s early-stage cloud business would not be included. Public reporting indicates the information originated from Alex Davis, CEO of Disruptive (lead investor in Groq’s latest financing), and that neither NVIDIA nor Groq had issued an immediate confirmation at the time of publication. The same reporting frames the transaction as coming together quickly, only months after Groq raised $750M at a ~$6.9B valuation, and highlights Groq’s positioning as a high-performance inference chip vendor founded by ex-Google TPU engineers. Groq is best understood as a vertically integrated inference acceleration company whose core asset is an application-specific processor optimized for deterministic, low-latency execution of transformer-style workloads, paired with a compiler-led software stack and a distribution layer (GroqCloud) designed to reduce developer friction via OpenAI-compatible APIs and integrations. Groq brands its architecture as a Language Processing Unit (LPU) and consistently emphasizes that the design target is inference, not training. The company’s own architecture description centers on 1-core execution, large on-chip SRAM used as primary storage (explicitly not cache), a custom compiler that statically schedules compute and communication, and direct chip-to-chip connectivity intended to coordinate multi-chip execution without relying on conventional caching hierarchies or dynamic runtime scheduling. The technical premise is a deliberate inversion of the conventional GPU approach. GPUs deliver throughput via massively parallel, multi-core execution with dynamic scheduling, complex memory hierarchies, and heavy reliance on off-chip HBM bandwidth and sophisticated runtime/kernel optimization. Groq instead argues that inference bottlenecks are driven by latency variance (tail latency), synchronization overhead, and memory access unpredictability inherent in dynamically scheduled, cache-heavy architectures, particularly when workloads are latency sensitive and batch sizes cannot be inflated. Groq’s solution is to move “control” into the compiler: the full execution graph and inter-chip communication schedule are computed ahead of time down to clock-cycle granularity, with deterministic execution designed to reduce run-to-run variance. In Groq’s framing, the removal of caches, reorder buffers, speculative execution overhead, and other sources of contention enables predictable latency and high utilization without per-model kernel engineering typical of GPU tuning cycles. A critical nuance is that Groq’s determinism is not merely a software claim; it is tightly coupled to architectural constraints and system design choices that trade flexibility for predictability. Third-party technical commentary indicates Groq’s chip uses a fully deterministic VLIW-style approach with minimal buffering, no external memory, and heavy dependence on sharding models across many chips because on-chip SRAM capacity is limited. SemiAnalysis describes a ~725 mm^2 die on GlobalFoundries 14nm with ~230MB of SRAM and notes that “no useful models” fit on a single chip, forcing multi-chip partitioning for modern LLMs and driving a system-level design where networking and compilation are first-class scheduling problems rather than ancillary infrastructure. This is consistent with Groq’s own messaging that tensor parallelism across chips is a primary design goal, enabled by large on-chip SRAM and compile-time coordination of compute plus interconnect. The on-chip SRAM emphasis is central to Groq’s latency story and also its most constraining trade-off. Groq claims on-chip SRAM bandwidth “upwards of 80 TB/s” and contrasts that with off-chip HBM bandwidth “about 8 TB/s,” asserting a potential 10x advantage from bandwidth plus reduced trips across chip-to-memory boundaries. While these comparisons are marketing-oriented and depend on workload specifics, the architectural implication is clear: Groq prioritizes ultra-fast local weight/activation access and then scales capacity by adding chips, not by attaching large off-chip memory pools. This design can reduce latency for sequential inference layers and minimize unpredictable stalls, but it pushes complexity into partitioning strategy, interconnect topology, and compiler scheduling, and it increases the number of chips needed for very large parameter counts and large KV-cache footprints. Groq also highlights numeric formats and compiler-driven precision management as a performance lever. In its 2025 technical blog, Groq describes “TruePoint numerics,” including 100-bit intermediate accumulation and selective quantization choices (FP32 for attention-sensitive operations, block floating point for MoE weights, FP8 storage in error-tolerant layers), and claims 2-4x speedups versus BF16 without measurable accuracy degradation on benchmarks such as MMLU and HumanEval. Even if the absolute uplift is workload dependent, the strategic point is that Groq is pursuing performance via end-to-end co-design: precision policy is not just hardware capability (FP8/BF16) but compiler-enforced mapping of precision to error sensitivity, which can matter materially for inference cost-per-token if it reduces memory traffic and boosts throughput without forcing aggressive, accuracy-damaging quantization. Independent performance datapoints indicate Groq has been credible on latency-oriented inference speed, at least for certain regimes. EE Times reported in 2023 that Groq demonstrated Llama-2 70B inference at ~240 tokens/s per user on a cloud-based dev system described as 10 racks and 64 chips, using the company’s 1st-gen silicon introduced several years earlier. Separate Groq commentary around independent benchmarking cites results showing ~241 tokens/s throughput and ~0.8s time to receive 100 output tokens for a Llama-2 70B API configuration, positioning the platform as a step-change in “available speed” for certain interactive use cases. These figures do not settle total cost-of-ownership versus GPUs or hyperscaler ASICs, but they establish that Groq’s system-level architecture can deliver strong single-user throughput and latency on large models when properly partitioned and scheduled. GroqCloud is the commercial wrapper that packages this hardware/software stack as “tokens-as-a-service,” aiming to make Groq adoption feel like switching API endpoints rather than adopting new silicon. Groq’s documentation states its API is designed to be “mostly compatible” with OpenAI client libraries, and its pricing page provides model-specific token rates, published speeds (tokens/s), prompt caching discounts, and batch processing discounts. For example, pricing lists inputs as low as $0.05 per 1M tokens and outputs as low as $0.08 per 1M tokens for certain smaller LLM configurations, with higher prices for larger models and long-context or MoE variants; it also advertises prompt caching with a 50% discount on cached input tokens for certain models and a batch API offering 50% lower cost for asynchronous processing windows. These mechanics are economically important because they demonstrate Groq’s go-to-market is not simply “sell chips,” but “sell predictable unit economics per token,” with tooling (batch, caching) that directly targets inference cost drivers (reused prompts, throughput smoothing, and asynchronous workloads). The cloud footprint and distribution partnerships indicate Groq has been building an inference-native “edge within the cloud” strategy rather than competing head-on with hyperscalers on breadth of services. A 2025 Groq newsroom release describes a European deployment in Helsinki with Equinix, positioned as latency reduction and data governance for European customers, and explicitly references Equinix Fabric enabling private connectivity to GroqCloud over public, private, or sovereign infrastructure. The same release enumerates additional capacity in the U.S. (Equinix, DataBank), Canada (Bell Canada), and Saudi Arabia (HUMAIN), and states these sites collectively served more than 20M tokens/s across Groq’s global network at that time. That supply-side metric matters because it provides a directional sense that Groq is scaling capacity as a network, not merely as a chip vendor. Customer disclosure is inherently limited because Groq is private and many enterprise deployments are not public, but Groq’s marketing materials and partnerships provide signals about demand vectors. The company’s public website displays logos of large consumer and enterprise brands (e.g., Dropbox, Vercel, Chevron, Volkswagen, Canva, Robinhood, Riot Games, Workday, Ramp) and includes a published customer quote claiming a 7.41x chat speed increase and an 89% cost reduction after moving to GroqCloud, followed by a tripling of token consumption. While marketing claims should be treated as case-specific and not generalized, they indicate that Groq is targeting both AI-native developers (who measure success by latency and cost-per-token) and enterprise buyers (who care about predictable performance and governance). Supplier and dependency mapping for Groq spans 3 layers: silicon production, system integration, and cloud infrastructure. On silicon, third-party analysis indicates GlobalFoundries 14nm for the 1st-gen Groq chip, implying a supply chain less constrained by the most capacity-tight leading-edge nodes and advanced packaging bottlenecks that dominate high-end GPU supply (HBM stacks, CoWoS-type packaging constraints). If accurate, this is strategically meaningful because it suggests Groq capacity expansion could be gated more by conventional wafer supply, board assembly, and data center power than by the same HBM/advanced packaging scarcity that has constrained top-tier GPU ramp cycles. On systems and cloud, Groq’s own releases identify colocation and connectivity partners (Equinix, DataBank, Bell Canada) and a Middle East partner (HUMAIN), implying dependencies on data center real estate, power availability, and network connectivity, alongside procurement of standard server components, NICs/switching, racks, and cooling infrastructure. The Groq design narrative also emphasizes air cooling and reduced need for complex power/cooling infrastructure, which—if realized in deployments—can widen the set of feasible hosting locations and lower deployment friction relative to liquid-cooled, very high power density GPU racks. Against that backdrop, the strategic rationale for NVIDIA acquiring Groq can be framed as a set of overlapping objectives: inference silicon optionality, architectural hedging, competitive defense, and supply chain diversification, with the carve-out of GroqCloud signaling a preference to avoid direct cloud competition and to focus on IP and product portfolio control rather than operating a capital-intensive token-serving business. The deal, if confirmed, would occur at a valuation step-up of ~190% versus Groq’s reported ~$6.9B private valuation in the September $750M round, reinforcing that any acquisition logic would be predominantly strategic rather than a conventional financial multiple arbitrage. The most compelling strategic driver is inference. Training has historically been the center of gravity for cutting-edge GPU demand, but inference volume is structurally larger and more distributed as deployments scale, with economics dominated by cost-per-token, latency guarantees, and utilization under spiky demand. Inference workloads also create a strategic vulnerability for NVIDIA: hyperscalers and large platforms can justify bespoke ASICs (TPU, Trainium/Inferentia, Maia-class efforts) because inference is stable, repeatable, and can amortize software investment at massive scale. Groq’s core proposition—deterministic, compiler-scheduled inference with predictable latency—aligns directly with the segment where GPU generality is least valued and where “good enough” programmability plus superior unit economics can win share. Acquiring Groq would allow NVIDIA to own a credible inference-native architecture rather than relying solely on GPUs and software optimization to defend that segment. Competitive defense logic is also plausible. Groq occupies a specific competitive wedge: low-latency, high-throughput interactive inference, delivered via a simple API abstraction that reduces switching cost. That wedge directly pressures GPU inference margins in the long run because it makes inference price/performance comparisons more transparent at the token level, and it targets a developer persona that historically defaulted to CUDA-first ecosystems. Even if NVIDIA’s current-generation systems can achieve very high tokens/s per user with extensive optimization, the strategic risk is that competing architectures normalize the idea that inference is best served by special-purpose silicon with a simpler programming model, weakening CUDA lock-in at the application layer. NVIDIA has actively demonstrated that Blackwell-era systems can exceed 1,000 tokens/s per user in benchmarked configurations, but that performance leadership does not automatically translate to lowest cost-per-token across the full range of batch sizes, latency targets, and deployment environments. Groq’s existence as a credible alternative architecture forces NVIDIA to keep defending inference economics rather than only raw performance leadership. The “technology acquisition” rationale is unusually strong in this specific case because Groq’s differentiator is not a single block of silicon IP but an end-to-end methodology: compiler-led static scheduling, deterministic networking, and a system architecture designed around tensor-parallel inference rather than throughput-maximizing batch inference. NVIDIA’s stack is already compiler-heavy (TensorRT, Triton, CUDA graphs, kernel fusion, speculative decoding techniques), but GPUs remain dynamically scheduled devices with complex memory hierarchies and stochastic latency behaviors under contention. Groq’s approach provides an alternate design point: treating the entire inference execution (compute plus communication) as a statically schedulable program. In principle, that IP could be valuable even if Groq silicon itself is not adopted at massive scale, because it can inform how NVIDIA builds future inference-optimized products, compilers, and networking fabrics, especially as distributed inference with large models makes communication a first-order performance determinant. Supply chain diversification is a non-obvious but potentially important driver. If Groq’s mainstream product generation is truly based on a mature process node and avoids HBM, then the scaling constraints look different than those of state-of-the-art GPUs. NVIDIA’s ability to meet incremental demand has been tightly coupled to advanced packaging and HBM supply, and those constraints can remain binding even when wafer supply is available. An inference ASIC architecture that relies primarily on on-chip SRAM and scales by adding chips—while not costless—could reduce dependence on HBM availability and advanced packaging capacity, enabling NVIDIA to ship “inference capacity” in higher absolute volumes or into geographies and customer segments where the highest-end GPUs are economically or logistically difficult to deploy. This could be particularly relevant for latency-sensitive inference deployed in regional colocation footprints rather than centralized hyperscale campuses. The carve-out of GroqCloud, if accurate, is itself a strategic signal about NVIDIA’s priorities. Operating a token-serving cloud at scale is capital intensive, structurally lower margin than silicon IP rents, and creates channel conflict with hyperscalers and CSP partners who are core NVIDIA customers. NVIDIA has generally positioned its cloud offerings through partnerships rather than as a direct hyperscale competitor. Excluding GroqCloud would preserve neutrality with CSPs and avoid inheriting multi-region data residency obligations and partner contracts, while still allowing NVIDIA to acquire Groq’s silicon, compiler technology, and engineering talent. At the same time, excluding GroqCloud would also mean NVIDIA would not automatically acquire the commercial proof-point of Groq’s unit economics or the customer contracts that validate product-market fit at scale, increasing the importance of diligence on whether Groq’s cloud pricing is structurally profitable or partially subsidized by fundraising. There is also a “preemptive acquisition” angle. The reporting identifies recent investors in Groq’s latest round including large financial institutions and strategic/industry players. In that context, Groq represents an asset that could plausibly have been acquired by a competitor (AMD/Intel) or by a hyperscaler seeking to accelerate inference independence. NVIDIA acquiring Groq could be a defensive move to prevent a credible inference-native architecture from being weaponized by a rival with deep distribution. Even if GroqCloud is carved out, controlling the silicon roadmap and compiler IP would meaningfully constrain Groq’s ability to evolve into a standalone competitor, unless the carved-out entity retains long-term rights to the hardware and software stack. However, the strategic case is not one-sided; there are meaningful risks and potential contradictions that would need to be reconciled for the transaction to be value-accretive on a multi-year horizon. 1st, Groq’s architecture appears to rely on scaling out chip count to achieve capacity, which introduces system cost, networking complexity, and physical footprint considerations. The absence of external memory and limited on-chip SRAM implies very large models require substantial chip parallelism, and the economics then depend heavily on chip cost, yield, power efficiency, and interconnect overhead. SemiAnalysis explicitly frames Groq as trading space for time and raises questions about token economics and whether publicly advertised pricing reflects fully loaded costs or market share capture. 2nd, integration risk is non-trivial. Groq’s compiler-led deterministic model is philosophically and practically different from CUDA’s dominant programming and execution model. A poorly executed integration could create internal product confusion, dilute engineering focus, or alienate developers if the combined stack fragments. 3rd, there is cannibalization risk. If Groq-class inference silicon undercuts GPU inference economics, NVIDIA could face internal margin trade-offs, even if the goal is to defend share against hyperscaler ASICs. Cannibalization can still be rational if it prevents larger share loss, but it would require crisp portfolio segmentation and go-to-market discipline. The presence of NVIDIA’s own rapidly improving inference performance complicates the “need” for Groq but does not eliminate the “option value.” NVIDIA has demonstrated benchmark-leading tokens/s per user on Blackwell-based systems, suggesting that raw interactive throughput is not necessarily the limiting factor for NVIDIA’s product line. The more enduring strategic question is unit economics and architectural control: whether future inference demand is better monetized through general-purpose GPUs plus software optimization, or whether a bifurcated product portfolio (training GPUs plus inference-native ASICs) becomes necessary to defend total AI compute wallet share as hyperscaler ASIC penetration increases. Acquiring Groq could be a decisive move to ensure NVIDIA participates in both regimes rather than betting exclusively on GPUs to win inference forever. What is “special” about Groq’s technology relative to a typical accelerator roadmap is the tight coupling of determinism, compilation, and networking into a single scheduling problem. The LPU narrative emphasizes deterministic compute and networking, static scheduling, and direct chip-to-chip coordination that allows “hundreds” (more precisely, 100s) of chips to behave like a single scheduled resource. The architecture also explicitly targets tensor-parallel, latency-optimized distribution rather than pure data-parallel throughput scaling, which matters for real-time applications where a single response must arrive quickly rather than many requests being processed in bulk. The implication is that Groq is optimized for the time-to-first-token and steady token streaming behavior that defines user experience in interactive LLMs, and it attempts to achieve that without relying on large batch sizes that can degrade latency. From a portfolio manager’s perspective, the most important interpretation is that an NVIDIA-Groq combination would likely be less about “NVIDIA needs more inference speed” and more about controlling the architectural trajectory of inference acceleration and removing a fast-improving, developer-friendly competitor from the market. The carve-out of GroqCloud would reinforce that the transaction is aimed at IP, talent, and product optionality, not acquiring a cloud revenue stream. The valuation step-up implied by $20B versus $6.9B would therefore be justified only if the acquired assets materially reduce long-term competitive risk (hyperscaler ASIC displacement, inference margin compression) or enable new monetization vectors (inference ASIC product line, supply chain de-bottlenecking, improved software determinism) that would be difficult to achieve on a comparable timeline via internal R&D.

TheValueist

102,145 görüntüleme • 9 ay önce

$NVDA $MU $SNDK $LITE PAPER OVERVIEW AND CORE CLAIMS The paper “KV Cache Transform Coding for Compact Storage in LLM Inference” introduces kvtc, a transform-coding pipeline that compresses transformer key-value (KV) caches primarily for storage and transfer in LLM serving, rather than for accelerating the per-token attention kernel during active decoding. The method combines 3 stages: (1) feature decorrelation via a PCA basis computed from a calibration dataset and reused across requests; (2) adaptive, variable-precision quantization with bit allocation solved via dynamic programming (DP), including groupwise scaling/shift overhead; and (3) lossless entropy coding (DEFLATE via nvCOMP in the reference implementation) to exploit residual redundancy after quantization. The central empirical claim is that KV tensors contain large, exploitable redundancy across heads and layers, enabling approximately 20× compression versus a 16-bit baseline with negligible degradation across a broad set of accuracy and long-context benchmarks, with materially higher compression (≥40×) available at modest quality cost in some regimes. The system claim is that such compression materially improves the economics of multi-turn, prefix-reuse serving by extending effective KV cache capacity in GPU HBM and host tiers (DRAM/NVMe) and by reducing inter-node and GPU↔host bandwidth demands, thereby improving cache hit rates and reducing time-to-first-token (TTFT) relative to recomputation when caches would otherwise be evicted. KV CACHE AS THE DOMINANT STATE VARIABLE IN INFERENCE ECONOMICS KV cache growth is linear in context length and is multiplicative in layers and attention heads, making it an increasingly dominant constraint as (a) context lengths expand, (b) models add layers and maintain large hidden dimensions, and (c) production workloads shift toward iterative and tool-augmented interactions that repeatedly reuse long prefixes. The paper uses the canonical 16-bit KV cache size formula (4·l·h·d_head·t) bytes and reports 16-bit KV cache sizes per 1K tokens of context that are already operationally large: 128MiB for Llama 3.1 8B, 160MiB for Mistral NeMo 12B, and 320MiB for Llama 3.3 70B Instruct. In binary units, these figures imply per-token KV footprints of 128KiB/token (Llama 3.1 8B), 160KiB/token (Mistral NeMo 12B), and 320KiB/token (Llama 3.3 70B Instruct) at 16-bit. For a 10K-token prompt (10×1K in the paper’s binary convention), the 16-bit KV cache sizes scale to approximately 1.25GiB (Llama 3.1 8B), 1.56GiB (Mistral NeMo 12B), and 3.13GiB (Llama 3.3 70B Instruct). These magnitudes explain why stale caches create a throughput–latency dilemma: retaining them in HBM maximizes responsiveness on future turns but crowds out concurrent sessions; evicting them forces quadratic-cost prefill recomputation and increases TTFT; offloading them to host or storage introduces large transfer overhead and consumes DRAM/NVMe capacity. A key operational nuance emphasized is that modern serving stacks increasingly treat KV caches as a database, leveraging block paging and shared-prefix reuse. In the common disaggregated serving design (separate prefill and decode nodes), KV cache transfer becomes a dominant category of cross-node traffic. Under that design, any reduction in KV cache size directly increases effective fabric capacity and reduces tail latency attributable to congestion, while also enabling longer cache lifetimes in “hot” (HBM) and “warm” (CPU DRAM) tiers that raise cache hit rates and reduce recomputation frequency. The paper’s quantitative example illustrates the economic stakes: a 1,000-line code file tokenized at ~10 tokens/line yields ~10K tokens; for Llama 3.3 70B, an 8-bit KV cache for that context is ~1.6GiB. Reuse across subsequent turns or parallel chats around the same file is valuable, but HBM scarcity makes retaining many such caches infeasible without compression. TECHNICAL MECHANISM: WHY KV CACHES ARE COMPRESSIBLE AND HOW KVTC EXPLOITS IT The technical rationale begins with an empirical observation: keys (and, to a lesser extent, values) across different attention heads can be aligned into a shared latent space using orthogonal transformations (Procrustes alignment). This supports the hypothesis that head-specific projections introduce rotations of a common subspace rather than completely distinct information, implying that concatenating across heads and layers should reveal low-rank structure suitable for linear decorrelation and dimensionality reduction. The method operationalizes this using a PCA/SVD basis learned from calibration data rather than recomputing a decomposition per prompt. This design choice targets production viability: per-prompt SVD is computationally expensive and scales poorly with long prompts and frequent cache updates. kvtc is explicitly structured as an offline-calibrated, online-applied codec: Calibration (performed 1 time per model and compression setting for DP allocation) A calibration dataset is forwarded through the model to collect KV caches. Token positions are pooled, and a subset of positions is sampled. Keys and values are processed separately. Several implementation choices are highlighted as decisive for stability: Rotary positional embeddings are effectively removed prior to compression (“undo positional rotations”), because positional rotations degrade the apparent low-rank structure of keys. “Attention sink” tokens (the earliest tokens in the sequence) and a sliding window of most recent tokens are excluded from compression because they disproportionately affect attention patterns and are empirically more sensitive to reconstruction error. Cross-layer concatenation is used: keys (or values) from multiple layers and heads at the same token position are concatenated along the feature axis to form a higher-dimensional feature vector. PCA is computed over these concatenated vectors, improving robustness relative to per-layer or per-head PCA. The PCA basis is computed via SVD of centered calibration data, using randomized SVD for scalability with a target rank cutoff. The paper reports calibration regimes of 160K tokens for several models with a 10K PCA dimension cutoff (8K for Qwen variants with fewer KV heads), selected to fit within a single 80GB H100 memory envelope and complete within minutes. A critical economic detail is that the same PCA basis can be reused across multiple compression ratios; only the DP-derived precision assignment changes per compression target. Compression (applied between inference phases) Compression operates on stored KV cache tensors, not on weights, and does not modify attention computation. The KV cache is projected into the PCA basis, quantized, packed, and then entropy-coded. Compression is positioned as a background or between-phase operation (after decoding, or between prefill and decode), executed on GPU or CPU depending on where the cache currently resides. The design intent is that compression should not sit on the critical per-token decoding path; it is a storage and transport optimization. Decompression (performed prior to reuse) Decompression reverses the entropy coding and quantization and applies the inverse PCA projection. A practical latency optimization is proposed: inverse projection can be performed layer-by-layer using submatrices of the PCA basis, allowing generation to begin before the full cache is reconstructed, reducing TTFT. Quantization and bit allocation are the core differentiators versus simpler PCA truncation. PCA provides ordered components by variance; kvtc uses DP to allocate a global bit budget across PCA coordinates (and across groups of coordinates) to minimize reconstruction error in the decorrelated domain. Groups of subsequent PCA coordinates share 16-bit shift and scale factors (a microscaling-inspired design), and the DP algorithm jointly selects group size and precision type under a bit budget, including the overhead of per-group metadata. DP commonly assigns 0 bits to many trailing PCA components, which both increases compression and provides a mechanism to trim the PCA basis to the subset of components that actually carry payload, reducing compute and storage overhead of the projection matrices in deployment. Lossless entropy coding then exploits the structure induced by quantization. DEFLATE is used in the reference implementation, and the paper emphasizes that the incremental gain from the lossless stage is content-dependent but meaningful, with an average uplift of ~1.23× on top of quantization in the reported regime. An ablation in the appendices indicates that GPU-friendly variants (GDeflate) can achieve nearly identical compression ratios (≤0.1 difference in measured cases), implying that throughput-optimized lossless codecs can likely be substituted without sacrificing meaningful compression. EMPIRICAL RESULTS: ACCURACY, COMPRESSION, AND LATENCY General-purpose 8B–12B dense models The paper evaluates Llama 3.1 8B, MN-Minitron 8B, and Mistral NeMo 12B across math/knowledge (GSM8K, MMLU) and long-context tasks (Qasper, Lost in the Middle, RULER Variable Tracking) under a simulated multi-turn regime where compression/decompression is applied periodically, with a sliding window of recent tokens excluded. A consistent pattern appears: kvtc maintains near-vanilla performance through 16× compression settings, and remains competitive at 32×, with degradation becoming task- and model-dependent at 64×, particularly on long-context retrieval metrics when compression is pushed aggressively. Selected quantitative anchor points from the paper’s standard-error table (all values are reported with the paper’s evaluation setup and token-window exclusions): Llama 3.1 8B Vanilla: GSM8K 56.8, MMLU 60.5, Qasper 40.4, LITM 99.4, RULER-VT 99.8 kvtc16×: GSM8K 56.9, MMLU 60.1, Qasper 40.7, LITM 99.3, RULER-VT 99.1 kvtc32×: GSM8K 57.8, MMLU 60.6, Qasper 39.4, LITM 99.1, RULER-VT 98.9 kvtc64×: GSM8K 57.2, MMLU 60.7, Qasper 37.8, LITM 90.2, RULER-VT 95.9 These results indicate that, for this model, long-context sensitivity emerges at 64× with meaningful drops in LITM and RULER-VT, while math/knowledge scores remain stable, implying a differential sensitivity consistent with key-vector precision being more critical for retrieval-style behavior. Mistral NeMo 12B Vanilla: GSM8K 61.9, MMLU 64.5, Qasper 38.4, LITM 99.5, RULER-VT 99.8 kvtc16×: GSM8K 62.0, MMLU 64.4, Qasper 37.6, LITM 99.8, RULER-VT 99.5 kvtc32×: GSM8K 62.2, MMLU 63.8, Qasper 37.5, LITM 99.6, RULER-VT 98.7 kvtc64×: GSM8K 61.9, MMLU 61.4, Qasper 38.0, LITM 95.3, RULER-VT 98.0 Here, degradation at 64× is visible but materially smaller than the Llama 3.1 8B LITM drop, suggesting model-architecture or training-data differences can change the tolerance envelope for aggressive KV cache distortion. MN-Minitron 8B Vanilla: GSM8K 59.1, MMLU 64.3, Qasper 38.2, LITM 99.8, RULER-VT 99.4 kvtc16×: GSM8K 60.3, MMLU 64.1, Qasper 38.6, LITM 99.3, RULER-VT 98.8 kvtc32×: GSM8K 59.1, MMLU 63.7, Qasper 37.7, LITM 86.9, RULER-VT 96.0 kvtc64×: GSM8K 57.8, MMLU 62.1, Qasper 38.1, LITM 59.5, RULER-VT 93.4 This model shows markedly higher sensitivity on LITM at 32× and 64×, despite stable short-context metrics, reinforcing that “compression safety” is not monotonic in parameter count and that pruning/distillation choices can alter KV cache redundancy or robustness. Comparisons to baselines The paper compares kvtc to quantization baselines (KIVI, GEAR, FP8) and eviction baselines (H2O, TOVA), plus an SVD-based prefill-optimization method (xKV). Across the reported tasks: Low-bit quantization methods at modest compression (2-bit KV schemes) show earlier degradation in long-context behavior than kvtc at substantially higher compression settings. Eviction methods perform poorly as generic compressors for long-context tasks, consistent with their objective function (selective pruning) being misaligned with “lossless-ish storage for reuse.” xKV shows competitive results on some tasks but a consistent underperformance on Qasper relative to kvtc and vanilla in the provided tables, consistent with method-specific distortions introduced by its decomposition regime. Reasoning models and high-variance tasks For DeepSeek-R1-distilled Qwen 2.5 reasoning models, the paper evaluates AIME 2024/2025 and LiveCodeBench coding. Results are averaged over 8 runs with large variance, but a key inference is that kvtc at ~9×–21× compression achieves broadly similar AIME scores within variance bands, while coding performance remains stable at ~9× and degrades more visibly at ~18×–21× on the 7B model. An important nuance is that smaller reasoning models already have smaller KV footprints (reported ~29KiB/token for Qwen R1 1.5B versus 131KiB/token for Llama 3.1 8B), so the economic value of aggressive KV cache compression is proportionally higher for large models and long contexts than for small models with short contexts, unless the serving system’s bottleneck is dominated by cache transfer rather than HBM capacity. Multi-GPU inference and pipeline parallel For Llama 3.3 70B Instruct run pipeline-parallel across 4 GPUs (20 layers per GPU), the paper compresses KV cache chunks independently per GPU. On MATH-500, the reported accuracy declines from 75.6 (vanilla) to 74.4 at 10× and 72.6 at 20×, with standard errors near ~1.9. NIAH and LITM remain at 100.0 for all tested ratios in that table. The paper notes that joint compression across chunks could improve accuracy for some offload scenarios but is not required for feasibility, highlighting an engineering trade-off between deployment simplicity in distributed settings and optimal global compression. Latency and TTFT economics A critical system result is the measured compression/decompression latency on an H100 for a non-fused implementation. For Mistral NeMo 12B in bfloat16: BS=8, CTX=8K: compression 379ms, decompression 267ms; vanilla recompute TTFT 3098ms; kvtc decompression TTFT 380ms BS=2, CTX=16K: compression 194ms, decompression 143ms; vanilla recompute TTFT 1780ms; kvtc decompression TTFT 208ms These measurements imply that, when a cache would otherwise be recomputed, decompressing a stored compressed cache can reduce TTFT by ~8×–9× in these scenarios, even without kernel fusion. The decomposition of runtime shows PCA projection and entropy coding as the largest contributors, implying that GPU-optimized kernels and faster GPU-native lossless codecs could reduce overhead further. The fundamental economic conclusion is that, in multi-turn settings with long prefixes, compression-induced overhead is likely dominated by the avoided prefill compute and avoided transfer overhead for uncompressed caches. KEY DEPLOYMENT-SENSITIVE DESIGN CHOICES AND FAILURE MODES Several design choices appear to be “hard requirements” rather than optional optimizations: Sink tokens and sliding window exclusions The paper’s ablations show that compressing early “sink” tokens can catastrophically degrade accuracy at high compression ratios (example: Llama 3.1 8B at 64× collapses on multiple tasks when sink tokens are compressed). Similarly, compressing the most recent tokens hurts performance, motivating a sliding window (default 128 tokens) that remains uncompressed. This introduces a predictable engineering constraint: kvtc is not a uniform compression of the full cache; it is a policy-driven, token-position-dependent codec. Production integration therefore requires correct handling of token positions, attention sinks, and window management, and these policies must be aligned with attention-kernel behavior and model-specific sink dynamics. RoPE handling Removing positional rotations prior to compression is described as important for preserving low-rank structure. In deployment, this implies that the codec must be position-aware and must invert and reapply RoPE correctly. This is an additional source of complexity relative to pure per-token quantization and is sensitive to model variants and RoPE parameterizations. Calibration set representativeness The method’s quality hinges on the PCA basis generalizing from calibration data to production data. The paper demonstrates relative stability with 160K–200K calibration tokens and explores domain shifts (general web text vs math traces vs code). Results suggest that moderate domain mismatch is tolerated at 16×–64×, while extreme compression (e.g., 256× in ablations) becomes materially more sensitive to calibration choice. In production, this implies that operators targeting the “negligible degradation” regime should be able to calibrate with broadly representative corpora, while operators targeting ultra-high compression for specialized workloads should expect tighter coupling between calibration domain and achieved quality. PCA matrix storage overhead and operational footprint A non-trivial hidden cost is the need to store PCA projection matrices per model. The paper reports that, prior to DP trimming, PCA matrices stored at 16-bit can amount to a meaningful fraction of model parameter count (examples reported: ~2.4% for Llama 3.3 70B, ~8.7% for Llama 3.1 8B). This overhead is amortized across all cached sessions for a model but competes with HBM/DRAM budgets in multi-model serving. DP-driven trimming can reduce this overhead at higher compression ratios by removing zero-bit components, but the directionality is not guaranteed at low compression ratios if many components remain active. In distributed inference (pipeline parallel), per-chunk PCA can reduce matrix sizes, but may reduce cross-layer decorrelation benefits if fewer layers are concatenated. SYSTEM-LEVEL IMPLICATIONS FOR GENERATIVE AI INFRASTRUCTURE GPU AND HBM The principal infrastructure implication is that KV cache compression at storage time targets the dominant memory allocator stressor in stateful serving: the accumulation of idle or warm conversation state. For workloads with long reusable prefixes (code assistants, enterprise agents with large system prompts, repeated RAG scaffolds, document chat), the limiting resource frequently becomes HBM reserved for KV caches rather than compute. By compressing stale caches by ~20× (or more), the same HBM budget can retain a materially larger working set of cached prefixes, increasing cache hit rates and reducing recomputation. This effect is multiplicative with cache-aware routing and prefix sharing: more prefixes can remain resident (hot or warm) and can be routed to nodes that already hold them, improving both throughput and tail latency. However, kvtc as described does not reduce the active KV cache footprint during the actual attention computation for a currently decoding sequence, because the model operates on decompressed KV caches during decoding. Therefore, the method does not directly reduce HBM bandwidth consumed by attention kernels during steady-state decode, and does not directly address the “memory traffic per generated token” bottleneck that motivates online KV quantization and eviction strategies. The primary HBM benefit is increased effective capacity for caches between turns and reduced HBM pressure from storing many idle sessions, not reduced per-token decode bandwidth. Compression and decompression themselves consume GPU compute and memory bandwidth. The measured decompression TTFT of ~208ms–380ms in the provided benchmarks indicates that the overhead is real but can be materially smaller than recomputation of long prefixes. In an HBM-constrained serving environment, this overhead can be interpreted as a trade between (a) maintaining more caches warm and paying decompression on reuse versus (b) evicting caches and paying full prefill recomputation. The decision boundary will depend on distribution of inter-turn idle times, probability of reuse, and SLA sensitivity to TTFT. kvtc expands the feasible region where keeping caches is economically rational, especially for long prompts. CPU AND DRAM The method implies a stronger role for CPU DRAM as a warm KV cache tier. A ~20× compression ratio changes the practical scale of “warm state” that can be stored per server. Using the paper’s reported KV cache sizes, a 10K-token 16-bit KV cache for Llama 3.3 70B is ~3.13GiB; compressing by ~20× would reduce this to ~160MiB. At that size, storing hundreds to thousands of warm conversation states in DRAM becomes materially more feasible, increasing cache hit rates and reducing NVMe dependence. This can shift system design from “HBM-only hot caches with aggressive eviction” toward “HBM hot + DRAM warm with long retention,” which is structurally analogous to CPU page cache hierarchies in classical systems design. CPU compute implications depend on where compression is executed. The paper explicitly allows compression on CPU if the cache is already in storage, but the strongest bandwidth savings are achieved when compression happens before moving KV caches off the GPU. If an operator chooses GPU-side compression prior to PCIe/NVLink transfer, CPU compute overhead is modest (orchestrating and DP calibration offline). If an operator instead transfers uncompressed caches to CPU for compression, bandwidth savings are forfeited and CPU memory bandwidth becomes a bottleneck. Therefore, the most economically coherent deployment path is GPU-native compression/decompression with CPU DRAM used as the warm storage reservoir.

$NVDA $MU $SNDK $LITE PAPER OVERVIEW AND CORE CLAIMS The paper “KV Cache Transform Coding for Compact Storage in LLM Inference” introduces kvtc, a transform-coding pipeline that compresses transformer key-value (KV) caches primarily for storage and transfer in LLM serving, rather than for accelerating the per-token attention kernel during active decoding. The method combines 3 stages: (1) feature decorrelation via a PCA basis computed from a calibration dataset and reused across requests; (2) adaptive, variable-precision quantization with bit allocation solved via dynamic programming (DP), including groupwise scaling/shift overhead; and (3) lossless entropy coding (DEFLATE via nvCOMP in the reference implementation) to exploit residual redundancy after quantization. The central empirical claim is that KV tensors contain large, exploitable redundancy across heads and layers, enabling approximately 20× compression versus a 16-bit baseline with negligible degradation across a broad set of accuracy and long-context benchmarks, with materially higher compression (≥40×) available at modest quality cost in some regimes. The system claim is that such compression materially improves the economics of multi-turn, prefix-reuse serving by extending effective KV cache capacity in GPU HBM and host tiers (DRAM/NVMe) and by reducing inter-node and GPU↔host bandwidth demands, thereby improving cache hit rates and reducing time-to-first-token (TTFT) relative to recomputation when caches would otherwise be evicted. KV CACHE AS THE DOMINANT STATE VARIABLE IN INFERENCE ECONOMICS KV cache growth is linear in context length and is multiplicative in layers and attention heads, making it an increasingly dominant constraint as (a) context lengths expand, (b) models add layers and maintain large hidden dimensions, and (c) production workloads shift toward iterative and tool-augmented interactions that repeatedly reuse long prefixes. The paper uses the canonical 16-bit KV cache size formula (4·l·h·d_head·t) bytes and reports 16-bit KV cache sizes per 1K tokens of context that are already operationally large: 128MiB for Llama 3.1 8B, 160MiB for Mistral NeMo 12B, and 320MiB for Llama 3.3 70B Instruct. In binary units, these figures imply per-token KV footprints of 128KiB/token (Llama 3.1 8B), 160KiB/token (Mistral NeMo 12B), and 320KiB/token (Llama 3.3 70B Instruct) at 16-bit. For a 10K-token prompt (10×1K in the paper’s binary convention), the 16-bit KV cache sizes scale to approximately 1.25GiB (Llama 3.1 8B), 1.56GiB (Mistral NeMo 12B), and 3.13GiB (Llama 3.3 70B Instruct). These magnitudes explain why stale caches create a throughput–latency dilemma: retaining them in HBM maximizes responsiveness on future turns but crowds out concurrent sessions; evicting them forces quadratic-cost prefill recomputation and increases TTFT; offloading them to host or storage introduces large transfer overhead and consumes DRAM/NVMe capacity. A key operational nuance emphasized is that modern serving stacks increasingly treat KV caches as a database, leveraging block paging and shared-prefix reuse. In the common disaggregated serving design (separate prefill and decode nodes), KV cache transfer becomes a dominant category of cross-node traffic. Under that design, any reduction in KV cache size directly increases effective fabric capacity and reduces tail latency attributable to congestion, while also enabling longer cache lifetimes in “hot” (HBM) and “warm” (CPU DRAM) tiers that raise cache hit rates and reduce recomputation frequency. The paper’s quantitative example illustrates the economic stakes: a 1,000-line code file tokenized at ~10 tokens/line yields ~10K tokens; for Llama 3.3 70B, an 8-bit KV cache for that context is ~1.6GiB. Reuse across subsequent turns or parallel chats around the same file is valuable, but HBM scarcity makes retaining many such caches infeasible without compression. TECHNICAL MECHANISM: WHY KV CACHES ARE COMPRESSIBLE AND HOW KVTC EXPLOITS IT The technical rationale begins with an empirical observation: keys (and, to a lesser extent, values) across different attention heads can be aligned into a shared latent space using orthogonal transformations (Procrustes alignment). This supports the hypothesis that head-specific projections introduce rotations of a common subspace rather than completely distinct information, implying that concatenating across heads and layers should reveal low-rank structure suitable for linear decorrelation and dimensionality reduction. The method operationalizes this using a PCA/SVD basis learned from calibration data rather than recomputing a decomposition per prompt. This design choice targets production viability: per-prompt SVD is computationally expensive and scales poorly with long prompts and frequent cache updates. kvtc is explicitly structured as an offline-calibrated, online-applied codec: Calibration (performed 1 time per model and compression setting for DP allocation) A calibration dataset is forwarded through the model to collect KV caches. Token positions are pooled, and a subset of positions is sampled. Keys and values are processed separately. Several implementation choices are highlighted as decisive for stability: Rotary positional embeddings are effectively removed prior to compression (“undo positional rotations”), because positional rotations degrade the apparent low-rank structure of keys. “Attention sink” tokens (the earliest tokens in the sequence) and a sliding window of most recent tokens are excluded from compression because they disproportionately affect attention patterns and are empirically more sensitive to reconstruction error. Cross-layer concatenation is used: keys (or values) from multiple layers and heads at the same token position are concatenated along the feature axis to form a higher-dimensional feature vector. PCA is computed over these concatenated vectors, improving robustness relative to per-layer or per-head PCA. The PCA basis is computed via SVD of centered calibration data, using randomized SVD for scalability with a target rank cutoff. The paper reports calibration regimes of 160K tokens for several models with a 10K PCA dimension cutoff (8K for Qwen variants with fewer KV heads), selected to fit within a single 80GB H100 memory envelope and complete within minutes. A critical economic detail is that the same PCA basis can be reused across multiple compression ratios; only the DP-derived precision assignment changes per compression target. Compression (applied between inference phases) Compression operates on stored KV cache tensors, not on weights, and does not modify attention computation. The KV cache is projected into the PCA basis, quantized, packed, and then entropy-coded. Compression is positioned as a background or between-phase operation (after decoding, or between prefill and decode), executed on GPU or CPU depending on where the cache currently resides. The design intent is that compression should not sit on the critical per-token decoding path; it is a storage and transport optimization. Decompression (performed prior to reuse) Decompression reverses the entropy coding and quantization and applies the inverse PCA projection. A practical latency optimization is proposed: inverse projection can be performed layer-by-layer using submatrices of the PCA basis, allowing generation to begin before the full cache is reconstructed, reducing TTFT. Quantization and bit allocation are the core differentiators versus simpler PCA truncation. PCA provides ordered components by variance; kvtc uses DP to allocate a global bit budget across PCA coordinates (and across groups of coordinates) to minimize reconstruction error in the decorrelated domain. Groups of subsequent PCA coordinates share 16-bit shift and scale factors (a microscaling-inspired design), and the DP algorithm jointly selects group size and precision type under a bit budget, including the overhead of per-group metadata. DP commonly assigns 0 bits to many trailing PCA components, which both increases compression and provides a mechanism to trim the PCA basis to the subset of components that actually carry payload, reducing compute and storage overhead of the projection matrices in deployment. Lossless entropy coding then exploits the structure induced by quantization. DEFLATE is used in the reference implementation, and the paper emphasizes that the incremental gain from the lossless stage is content-dependent but meaningful, with an average uplift of ~1.23× on top of quantization in the reported regime. An ablation in the appendices indicates that GPU-friendly variants (GDeflate) can achieve nearly identical compression ratios (≤0.1 difference in measured cases), implying that throughput-optimized lossless codecs can likely be substituted without sacrificing meaningful compression. EMPIRICAL RESULTS: ACCURACY, COMPRESSION, AND LATENCY General-purpose 8B–12B dense models The paper evaluates Llama 3.1 8B, MN-Minitron 8B, and Mistral NeMo 12B across math/knowledge (GSM8K, MMLU) and long-context tasks (Qasper, Lost in the Middle, RULER Variable Tracking) under a simulated multi-turn regime where compression/decompression is applied periodically, with a sliding window of recent tokens excluded. A consistent pattern appears: kvtc maintains near-vanilla performance through 16× compression settings, and remains competitive at 32×, with degradation becoming task- and model-dependent at 64×, particularly on long-context retrieval metrics when compression is pushed aggressively. Selected quantitative anchor points from the paper’s standard-error table (all values are reported with the paper’s evaluation setup and token-window exclusions): Llama 3.1 8B Vanilla: GSM8K 56.8, MMLU 60.5, Qasper 40.4, LITM 99.4, RULER-VT 99.8 kvtc16×: GSM8K 56.9, MMLU 60.1, Qasper 40.7, LITM 99.3, RULER-VT 99.1 kvtc32×: GSM8K 57.8, MMLU 60.6, Qasper 39.4, LITM 99.1, RULER-VT 98.9 kvtc64×: GSM8K 57.2, MMLU 60.7, Qasper 37.8, LITM 90.2, RULER-VT 95.9 These results indicate that, for this model, long-context sensitivity emerges at 64× with meaningful drops in LITM and RULER-VT, while math/knowledge scores remain stable, implying a differential sensitivity consistent with key-vector precision being more critical for retrieval-style behavior. Mistral NeMo 12B Vanilla: GSM8K 61.9, MMLU 64.5, Qasper 38.4, LITM 99.5, RULER-VT 99.8 kvtc16×: GSM8K 62.0, MMLU 64.4, Qasper 37.6, LITM 99.8, RULER-VT 99.5 kvtc32×: GSM8K 62.2, MMLU 63.8, Qasper 37.5, LITM 99.6, RULER-VT 98.7 kvtc64×: GSM8K 61.9, MMLU 61.4, Qasper 38.0, LITM 95.3, RULER-VT 98.0 Here, degradation at 64× is visible but materially smaller than the Llama 3.1 8B LITM drop, suggesting model-architecture or training-data differences can change the tolerance envelope for aggressive KV cache distortion. MN-Minitron 8B Vanilla: GSM8K 59.1, MMLU 64.3, Qasper 38.2, LITM 99.8, RULER-VT 99.4 kvtc16×: GSM8K 60.3, MMLU 64.1, Qasper 38.6, LITM 99.3, RULER-VT 98.8 kvtc32×: GSM8K 59.1, MMLU 63.7, Qasper 37.7, LITM 86.9, RULER-VT 96.0 kvtc64×: GSM8K 57.8, MMLU 62.1, Qasper 38.1, LITM 59.5, RULER-VT 93.4 This model shows markedly higher sensitivity on LITM at 32× and 64×, despite stable short-context metrics, reinforcing that “compression safety” is not monotonic in parameter count and that pruning/distillation choices can alter KV cache redundancy or robustness. Comparisons to baselines The paper compares kvtc to quantization baselines (KIVI, GEAR, FP8) and eviction baselines (H2O, TOVA), plus an SVD-based prefill-optimization method (xKV). Across the reported tasks: Low-bit quantization methods at modest compression (2-bit KV schemes) show earlier degradation in long-context behavior than kvtc at substantially higher compression settings. Eviction methods perform poorly as generic compressors for long-context tasks, consistent with their objective function (selective pruning) being misaligned with “lossless-ish storage for reuse.” xKV shows competitive results on some tasks but a consistent underperformance on Qasper relative to kvtc and vanilla in the provided tables, consistent with method-specific distortions introduced by its decomposition regime. Reasoning models and high-variance tasks For DeepSeek-R1-distilled Qwen 2.5 reasoning models, the paper evaluates AIME 2024/2025 and LiveCodeBench coding. Results are averaged over 8 runs with large variance, but a key inference is that kvtc at ~9×–21× compression achieves broadly similar AIME scores within variance bands, while coding performance remains stable at ~9× and degrades more visibly at ~18×–21× on the 7B model. An important nuance is that smaller reasoning models already have smaller KV footprints (reported ~29KiB/token for Qwen R1 1.5B versus 131KiB/token for Llama 3.1 8B), so the economic value of aggressive KV cache compression is proportionally higher for large models and long contexts than for small models with short contexts, unless the serving system’s bottleneck is dominated by cache transfer rather than HBM capacity. Multi-GPU inference and pipeline parallel For Llama 3.3 70B Instruct run pipeline-parallel across 4 GPUs (20 layers per GPU), the paper compresses KV cache chunks independently per GPU. On MATH-500, the reported accuracy declines from 75.6 (vanilla) to 74.4 at 10× and 72.6 at 20×, with standard errors near ~1.9. NIAH and LITM remain at 100.0 for all tested ratios in that table. The paper notes that joint compression across chunks could improve accuracy for some offload scenarios but is not required for feasibility, highlighting an engineering trade-off between deployment simplicity in distributed settings and optimal global compression. Latency and TTFT economics A critical system result is the measured compression/decompression latency on an H100 for a non-fused implementation. For Mistral NeMo 12B in bfloat16: BS=8, CTX=8K: compression 379ms, decompression 267ms; vanilla recompute TTFT 3098ms; kvtc decompression TTFT 380ms BS=2, CTX=16K: compression 194ms, decompression 143ms; vanilla recompute TTFT 1780ms; kvtc decompression TTFT 208ms These measurements imply that, when a cache would otherwise be recomputed, decompressing a stored compressed cache can reduce TTFT by ~8×–9× in these scenarios, even without kernel fusion. The decomposition of runtime shows PCA projection and entropy coding as the largest contributors, implying that GPU-optimized kernels and faster GPU-native lossless codecs could reduce overhead further. The fundamental economic conclusion is that, in multi-turn settings with long prefixes, compression-induced overhead is likely dominated by the avoided prefill compute and avoided transfer overhead for uncompressed caches. KEY DEPLOYMENT-SENSITIVE DESIGN CHOICES AND FAILURE MODES Several design choices appear to be “hard requirements” rather than optional optimizations: Sink tokens and sliding window exclusions The paper’s ablations show that compressing early “sink” tokens can catastrophically degrade accuracy at high compression ratios (example: Llama 3.1 8B at 64× collapses on multiple tasks when sink tokens are compressed). Similarly, compressing the most recent tokens hurts performance, motivating a sliding window (default 128 tokens) that remains uncompressed. This introduces a predictable engineering constraint: kvtc is not a uniform compression of the full cache; it is a policy-driven, token-position-dependent codec. Production integration therefore requires correct handling of token positions, attention sinks, and window management, and these policies must be aligned with attention-kernel behavior and model-specific sink dynamics. RoPE handling Removing positional rotations prior to compression is described as important for preserving low-rank structure. In deployment, this implies that the codec must be position-aware and must invert and reapply RoPE correctly. This is an additional source of complexity relative to pure per-token quantization and is sensitive to model variants and RoPE parameterizations. Calibration set representativeness The method’s quality hinges on the PCA basis generalizing from calibration data to production data. The paper demonstrates relative stability with 160K–200K calibration tokens and explores domain shifts (general web text vs math traces vs code). Results suggest that moderate domain mismatch is tolerated at 16×–64×, while extreme compression (e.g., 256× in ablations) becomes materially more sensitive to calibration choice. In production, this implies that operators targeting the “negligible degradation” regime should be able to calibrate with broadly representative corpora, while operators targeting ultra-high compression for specialized workloads should expect tighter coupling between calibration domain and achieved quality. PCA matrix storage overhead and operational footprint A non-trivial hidden cost is the need to store PCA projection matrices per model. The paper reports that, prior to DP trimming, PCA matrices stored at 16-bit can amount to a meaningful fraction of model parameter count (examples reported: ~2.4% for Llama 3.3 70B, ~8.7% for Llama 3.1 8B). This overhead is amortized across all cached sessions for a model but competes with HBM/DRAM budgets in multi-model serving. DP-driven trimming can reduce this overhead at higher compression ratios by removing zero-bit components, but the directionality is not guaranteed at low compression ratios if many components remain active. In distributed inference (pipeline parallel), per-chunk PCA can reduce matrix sizes, but may reduce cross-layer decorrelation benefits if fewer layers are concatenated. SYSTEM-LEVEL IMPLICATIONS FOR GENERATIVE AI INFRASTRUCTURE GPU AND HBM The principal infrastructure implication is that KV cache compression at storage time targets the dominant memory allocator stressor in stateful serving: the accumulation of idle or warm conversation state. For workloads with long reusable prefixes (code assistants, enterprise agents with large system prompts, repeated RAG scaffolds, document chat), the limiting resource frequently becomes HBM reserved for KV caches rather than compute. By compressing stale caches by ~20× (or more), the same HBM budget can retain a materially larger working set of cached prefixes, increasing cache hit rates and reducing recomputation. This effect is multiplicative with cache-aware routing and prefix sharing: more prefixes can remain resident (hot or warm) and can be routed to nodes that already hold them, improving both throughput and tail latency. However, kvtc as described does not reduce the active KV cache footprint during the actual attention computation for a currently decoding sequence, because the model operates on decompressed KV caches during decoding. Therefore, the method does not directly reduce HBM bandwidth consumed by attention kernels during steady-state decode, and does not directly address the “memory traffic per generated token” bottleneck that motivates online KV quantization and eviction strategies. The primary HBM benefit is increased effective capacity for caches between turns and reduced HBM pressure from storing many idle sessions, not reduced per-token decode bandwidth. Compression and decompression themselves consume GPU compute and memory bandwidth. The measured decompression TTFT of ~208ms–380ms in the provided benchmarks indicates that the overhead is real but can be materially smaller than recomputation of long prefixes. In an HBM-constrained serving environment, this overhead can be interpreted as a trade between (a) maintaining more caches warm and paying decompression on reuse versus (b) evicting caches and paying full prefill recomputation. The decision boundary will depend on distribution of inter-turn idle times, probability of reuse, and SLA sensitivity to TTFT. kvtc expands the feasible region where keeping caches is economically rational, especially for long prompts. CPU AND DRAM The method implies a stronger role for CPU DRAM as a warm KV cache tier. A ~20× compression ratio changes the practical scale of “warm state” that can be stored per server. Using the paper’s reported KV cache sizes, a 10K-token 16-bit KV cache for Llama 3.3 70B is ~3.13GiB; compressing by ~20× would reduce this to ~160MiB. At that size, storing hundreds to thousands of warm conversation states in DRAM becomes materially more feasible, increasing cache hit rates and reducing NVMe dependence. This can shift system design from “HBM-only hot caches with aggressive eviction” toward “HBM hot + DRAM warm with long retention,” which is structurally analogous to CPU page cache hierarchies in classical systems design. CPU compute implications depend on where compression is executed. The paper explicitly allows compression on CPU if the cache is already in storage, but the strongest bandwidth savings are achieved when compression happens before moving KV caches off the GPU. If an operator chooses GPU-side compression prior to PCIe/NVLink transfer, CPU compute overhead is modest (orchestrating and DP calibration offline). If an operator instead transfers uncompressed caches to CPU for compression, bandwidth savings are forfeited and CPU memory bandwidth becomes a bottleneck. Therefore, the most economically coherent deployment path is GPU-native compression/decompression with CPU DRAM used as the warm storage reservoir.

TheValueist

16,549 görüntüleme • 7 ay önce

BEARISH ON OPENAI The investment case for OpenAI has never been more precarious than it is right now in late 2025. What was once a company that seemed destined to dominate the artificial intelligence revolution has revealed itself to be a structurally disadvantaged challenger fighting a defensive war on multiple fronts. The company anticipates burning through roughly $9 billion this year on $13 billion in sales, a cash burn rate of approximately 70% of revenue. This is not the profile of a company poised to capture monopolistic profits from a transformative technology; it is the profile of a utility company spending astronomical sums to deliver a commodity product that competitors are increasingly giving away for free. The financial trajectory only becomes more alarming when examined over a longer time horizon. The documents show OpenAI projects that by 2028, its operating losses will balloon to roughly three-quarters of that year’s revenue, driven primarily by ballooning spending on computing costs. The company has painted a rosy picture of eventual profitability by 2029 or 2030, but this projection requires believing that OpenAI can grow revenue from roughly $13 billion today to $125 billion or more while simultaneously maintaining pricing power in a market where every major technology company and numerous startups are racing to commoditize the very product OpenAI sells. The cash burn is expected to reach $115 billion cumulatively through 2029, according to The Information. These numbers represent a staggering bet that requires near-perfect execution across multiple dimensions over half a decade. The most damning evidence against OpenAI’s long-term viability is the evaporation of its technological moat. In 2023, GPT-4 felt like genuine magic, a capability that no other company could replicate. Today, that lead has effectively vanished. The sudden availability of frontier-level open-source models is expected to dramatically accelerate AI development globally, potentially reshaping entire industries and altering the balance of power in the tech world. Meta’s Llama series, Mistral’s increasingly capable models, and even Chinese competitors like DeepSeek have demonstrated that the core technology powering ChatGPT is replicable and, in many cases, distributable for free. When your product becomes commoditized, the economics become brutal, and OpenAI finds itself in the position of trying to sell bottled water in a world where tap water has become indistinguishable in quality. The competitive pressure from open-source alternatives is compounding rapidly. The open source movement in AI has grown exponentially over the past few years. Instead of relying solely on expensive, closed models from major tech companies, developers and researchers worldwide can now access, modify, and improve upon state-of-the-art LLMs. This democratization is existential for OpenAI’s business model. Enterprises that once paid premium prices for API access now have the option to run comparable models on their own infrastructure at a fraction of the cost, with the added benefits of data privacy and customization. The value proposition that justified OpenAI’s premium pricing has eroded faster than anyone anticipated, and there is no indication that this trend will reverse. Perhaps nothing illustrates OpenAI’s structural weakness more clearly than the behavior of its most important partner. Microsoft is dancing to its own tune in the artificial intelligence revolution, and Wall Street cannot stop watching. Despite pouring approximately $13 billion into OpenAI over several years, DA Davidson analyst Gil Luria estimates that just 17 percent of Microsoft’s total Azure revenue comes from artificial intelligence workloads. More critically, only 6 percent of that total ties directly to reselling OpenAI’s models, while approximately 75 percent is generated from Azure AI. Microsoft is building its own models, hedging with Anthropic, and quietly reducing its dependency on the very company it funded. When your largest investor is simultaneously your biggest competitor and is actively developing alternatives to your core product, the strategic implications are dire. Leaders at Microsoft believe Anthropic’s latest models — Claude Sonnet 4, specifically — perform better than OpenAI’s in certain functions, like creating aesthetically pleasing PowerPoint presentations. This is not a minor technical preference; it represents a fundamental shift in how Microsoft views its partnership with OpenAI. Microsoft is dramatically escalating its AI independence strategy. At an internal town hall Thursday, Microsoft AI chief Mustafa Suleyman revealed the company is making “significant investments” in compute capacity to build frontier models that can compete directly with OpenAI, Google, and Meta. The company that was supposed to be OpenAI’s path to distribution and scale is instead preparing for a future where OpenAI is just one vendor among many, if not an outright competitor. The leadership exodus at OpenAI over the past year has been nothing short of catastrophic. In September 2024, Murati announced that she was stepping down as CTO. This move came amid a wider executive exodus as OpenAI chief research officer Bob McGrew and a vice president of research, Barret Zoph, also announced their departures soon after. Mira Murati was not a minor figure; she was instrumental in the development of ChatGPT, Dall-E, and Sora. Her departure, along with co-founder Ilya Sutskever, safety leader Jan Leike, and co-founder John Schulman who joined rival Anthropic, has left CEO Sam Altman without much of the leadership team that helped him build OpenAI into an AI juggernaut. Hannah Wong, the executive who steered OpenAI through its most chaotic period, has announced she’s leaving the company just this month, continuing the pattern of senior departures that suggests something fundamentally broken in the organization’s culture or direction. The distribution problem facing OpenAI may be its most insurmountable challenge. Apple and Google control the smartphones that billions of people use every day. Microsoft controls the productivity software that enterprises depend upon. OpenAI, by contrast, must convince users to deliberately open a separate application and type their queries into a text box. In a world of agentic AI where assistants need access to your email, calendar, and files to be useful, an AI embedded directly into your operating system has an overwhelming structural advantage over a standalone chatbot. OpenAI is trying to be a consumer product company without owning any of the surfaces where consumers actually spend their time, competing against incumbents who can simply bundle AI capabilities directly into products that already have hundreds of millions of daily active users. The nuclear-to-solar analogy captures the fundamental economic transformation that is devastating OpenAI’s business model. Just as nuclear power required enormous upfront capital expenditure for centralized power plants, AI in its current form requires massive data center investments to train and serve models. But the direction of travel is unmistakably toward distributed intelligence that runs locally on devices. A major part of the pitch is practicality. Lample emphasizes that Ministral 3 can run on a single GPU, making it deployable on affordable hardware — from on-premise servers to laptops, robots, and other edge devices that may have limited connectivity. When powerful AI models can run on a smartphone or a laptop without any cloud connection, the entire economic rationale for paying premium prices to access centralized AI infrastructure disappears. OpenAI is building nuclear reactors in a world that is rapidly installing solar panels on every rooftop. The proposed $1 trillion IPO valuation is perhaps the clearest signal that something is deeply wrong with the OpenAI story. In the first half of the year, OpenAI lost $13.5 billion, on revenue of $4.3 billion. It is on track to lose $27 billion for the year. One estimate shows OpenAI will burn $115 billion by 2029. Asking public market investors to pay $1 trillion for a company that loses more than twice as much as it earns is not a growth story; it is an exit strategy. The sophisticated investors who funded OpenAI’s private rounds are looking for a way to transfer their risk to retail investors and pension funds who may not fully understand the unit economics of the business. A recent report by HSBC estimated that the company will remain in the unprofitable category until 2029 and that the company will need an additional $207 billion to fund its ambitions. Sam Altman’s leadership represents another structural liability for the company. His background is as a startup investor and evangelist, not as an operational executive who has scaled a capital-intensive industrial operation. The pivot from nonprofit research lab to for-profit corporation to public benefit corporation to anticipated public company has been accompanied by legal and governance structures designed primarily to protect Altman’s control rather than to create shareholder value. Going public means answering a lot more of those kinds of questions, every single quarter, forever. When asked about financial concerns in a friendly podcast interview, Altman’s dismissive response revealed a leader uncomfortable with the scrutiny that public markets will inevitably bring. The adults in the room have largely departed, leaving a company that desperately needs disciplined execution led by someone whose strengths lie elsewhere. The comparison to Netscape is instructive. Netscape proved that the internet was real and created genuine value, but it had no sustainable moat against an incumbent who could bundle the browser directly into the operating system. OpenAI has proven that large language models are real and valuable, but it faces the same structural disadvantage against incumbents who can bundle AI directly into operating systems, productivity suites, and cloud platforms. The value will accrue to the companies that own the distribution channels and the hardware, not to the company that demonstrated the technology was possible. OpenAI is destined to become a historical footnote, remembered as the company that ignited the AI revolution but failed to capture the economic value it created. The only bull case for OpenAI is the AGI lottery ticket: the possibility that the company achieves artificial general intelligence before anyone else and thereby transcends all normal economic analysis. But there is no evidence that OpenAI is any closer to AGI than Google, Anthropic, or DeepMind. The company’s advantage was never secret research breakthroughs; it was first-mover advantage in commercialization. That advantage has now been erased by competitors who can match or exceed OpenAI’s capabilities while benefiting from existing ecosystems, distribution channels, and the willingness to operate AI as a loss leader to drive engagement with more profitable products. The secret sauce was never secret, and there was never any sauce. The endgame for OpenAI is unlikely to be the triumphant dominance that early investors imagined. The most probable outcomes range from gradual irrelevance as a backend provider, to financial restructuring under pressure from creditors, to absorption by Microsoft or another well-capitalized technology company looking to acquire the remaining talent and intellectual property at a discount. Despite its current losses, OpenAI’s long-term prospects are bolstered by the explosive growth of the AI market. But growth in the overall AI market does not guarantee success for any individual company, particularly one with no moat, no ecosystem, and a cost structure that requires selling a commodity at premium prices. The AI revolution is real, but OpenAI’s role in capturing its economic value is far from assured. For anyone considering an investment in OpenAI at anything close to current valuations, the prudent course is to stay far away and watch from the sidelines as economic reality catches up with hype.

David Shapiro (L/0)

69,180 görüntüleme • 9 ay önce

$MU $SNDK $LITE $VRT NVIDIA and Groq: 2nd and 3rd Order Strategic Infrastructure Effects and Market Implications Public reporting indicates NVIDIA has agreed to acquire Groq for approximately $20,000,000,000 in cash, while excluding Groq’s nascent cloud business from the transaction perimeter. The reported carve-out materially constrains the immediate, direct linkage from the acquisition to incremental, NVIDIA-controlled data center capacity build-out because GroqCloud appears to be the principal channel through which Groq hardware is currently monetized at scale as a service. The infrastructure-market implications therefore depend primarily on post-close product strategy: whether NVIDIA (1) commercializes Groq silicon as a distinct inference product line and drives broad deployment through OEM/ODM channels and partners, (2) uses the acquisition mainly to absorb IP and talent while de-emphasizing standalone Groq hardware volumes, or (3) uses Groq technology to reshape NVIDIA’s own inference systems and networking roadmaps. The dominant transmission mechanism into memory, networking, and facility infrastructure markets is the degree to which NVIDIA shifts incremental inference deployments away from GPU architectures that are tightly coupled to external high-bandwidth memory (HBM) and toward Groq’s current architecture, which emphasizes large on-chip SRAM, deterministic compiler-scheduled execution, and direct chip-to-chip connectivity. Independent and company-published materials describe Groq’s current-generation approach as having no external memory, keeping weights and KV cache on-chip during processing, and requiring model sharding across multiple chips due to limited on-chip SRAM per device. That architectural choice is directionally HBM-negative on a per-accelerator basis and ambiguous for DRAM, NAND, networking, power, and cooling on a per-token basis because the design can reduce memory wall losses and tail-latency overhead while potentially increasing the number of chips and interconnect endpoints required to serve large models and long-context workloads. HBM implications are the most mechanically straightforward but should be framed as second-derivative rather than absolute. If Groq-class inference silicon meaningfully displaces NVIDIA GPU-based inference deployments, incremental HBM bit demand tied to inference growth could be reduced relative to a GPU-only baseline because Groq’s current approach does not appear to attach HBM stacks to each accelerator. However, current market structure suggests HBM remains supply-constrained and is being pulled by multiple vectors including continued GPU training scale and high-capacity inference configurations, with leading suppliers signaling tight conditions extending beyond 2026. In that environment, reduced inference-driven HBM intensity could primarily reallocate scarce HBM supply toward higher-end training and premium inference GPUs rather than creating an outright volume collapse, preserving high utilization of HBM capacity while potentially affecting the slope of pricing power and capacity expansion urgency over a multi-year horizon. The key downside scenario for the HBM complex would be a durable architectural bifurcation where “good-enough” inference shifts disproportionately to HBM-less ASICs across a broad swath of deployments (latency-sensitive, batch-1, cost-per-token optimized), while training remains GPU-HBM dominated; such a split would reduce the portion of future inference compute that naturally monetizes through HBM content and could compress the incremental HBM-per-AI-dollar ratio. The key upside/neutral scenario for HBM is that the supply chain remains fully allocated regardless, with NVIDIA using any “freed” HBM to ship more high-end GPUs into training and long-context inference, especially as roadmaps increase HBM per GPU, sustaining robust aggregate bit demand even if inference becomes more heterogeneous. Conventional DRAM implications split into 2 channels: (1) DRAM wafer capacity diversion into HBM and (2) DDR content per server in AI clusters. Supplier commentary indicates that AI-driven memory demand is supporting elevated DRAM markets more broadly, and HBM production is resource-intensive versus conventional DRAM, tightening supply for DDR products in parallel. A meaningful NVIDIA pivot to an inference architecture that reduces HBM dependence could, at the margin, ease the most acute HBM-driven bottlenecks and allow memory manufacturers more flexibility in balancing DRAM mix, which could be modestly DDR-positive on the supply side (less crowding-out) even if it is DDR-neutral or slightly negative on the demand side (if per-node CPU/DDR requirements decline due to more efficient accelerator utilization). The dominant practical outcome is likely that DDR demand remains supported by broad AI server proliferation and increasing memory footprints at the system level (CPUs, networking stacks, caching layers, retrieval-augmented pipelines), while HBM remains the premium profit pool; therefore, any HBM displacement that increases total server volumes could indirectly keep DDR demand resilient even if DDR per accelerator is not rising materially. NAND flash implications are comparatively indirect and volume-driven rather than architecture-driven. Inference clusters require SSD capacity for model storage, container images, logging, and increasingly for fast local retrieval indices and embedding stores, but the storage footprint per unit of compute is typically smaller than in training pipelines that stage large datasets and checkpoints. If NVIDIA uses Groq to lower inference cost and latency enough to expand the total number of inference deployment locations (regional colocation, enterprise on-prem, sovereign footprints), aggregate SSD attach could rise through geographic fragmentation and replication of model artifacts across more sites, even if per-site storage is modest. The NAND effect is therefore likely to be demand-broadening and mix-positive (datacenter SSDs) but not a primary swing factor versus the macro AI capex cycle and consumer/device cycles. Hard disk drive (HDD) markets should see negligible direct sensitivity because nearline HDD demand is driven by bulk storage and cloud archiving economics, while inference acceleration choices primarily reshape compute and network layers; any HDD benefit would be a tertiary function of overall data center square footage expansion rather than a direct consequence of Groq silicon displacing GPUs. Optical networking implications require separating (1) intra-cluster back-end fabrics that connect accelerators and (2) front-end / data center interconnect (DCI) that connects sites and regions. Groq’s own positioning and third-party reporting suggest scaling beyond a single node or rack relies on high-bandwidth fabrics and, in some described configurations, optical interconnect scaling across hundreds of chips. If NVIDIA commercializes Groq at scale, 2 offsetting forces emerge: lower cost-per-token and improved latency could expand inference throughput and drive more east-west traffic, increasing demand for high-speed switching and optics; conversely, if Groq delivers materially higher utilization and tokens per unit of network bandwidth for certain workloads, the network required per served token could decline. Public NVIDIA materials already indicate an aggressive photonics roadmap aimed at scaling AI factories, including co-packaged optics (CPO) switches and explicit collaboration with Coherent and Lumentum in the silicon photonics supply chain. That linkage is important because it suggests that, independent of Groq, NVIDIA is already pushing optics integration deeper into the switch package to reduce power and increase resiliency; Groq increases the strategic incentive to reduce network power and latency if inference becomes even more distributed and latency-sensitive. For Lumentum and Coherent specifically, the net implication is less about “more optics versus fewer optics” and more about a shift in optics form factor and value capture. Co-packaged optics can reduce reliance on pluggable transceivers in some switch architectures while increasing demand for integrated photonic engines, lasers, fiber attach, packaging processes, and component-level supply. NVIDIA’s own announcements explicitly position Coherent and Lumentum as collaborators in creating the integrated silicon/optics process and supply chain for photonics switches. If Groq accelerates the transition to very large-scale fabrics (more endpoints, higher port speeds, tighter power envelopes), that tends to pull forward CPO adoption and amplifies demand for the underlying photonics components even if the conventional pluggable module TAM is structurally pressured over time. If Groq instead pushes inference toward smaller, more localized pods (closer to users, more regional colocation), that can be optics-positive for DCI and metro connectivity because more sites must be interconnected at high bandwidth with low latency, favoring coherent optics and high-speed interconnect between facilities. The principal risk for optics suppliers is timing and margin structure: a faster move to NVIDIA-driven integrated photonics could concentrate bargaining power and compress margins for commoditized transceiver modules while favoring suppliers with differentiated lasers, integration capability, and qualification depth in NVIDIA’s CPO ecosystem. AEC and copper interconnect implications hinge on whether Groq deployment increases the density of short-reach links inside racks and rows. High-speed copper remains structurally advantaged at very short distances on cost, power, and serviceability, but reaches become constrained as lane speeds and aggregate bandwidth rise, creating a role for active electrical cables (AECs), retimers, and signal-conditioning silicon. Credo explicitly positions its AEC products as enabling reliable lossless 800G connectivity for AI clusters, and the company has highlighted participation at NVIDIA GTC with content focused on extending PCIe/CXL using AECs, indicating relevance to next-generation system topologies that require longer reach and higher signal integrity than passive copper can deliver. If NVIDIA turns Groq into a widely deployed inference card or chassis product, the likely near-term effect is AEC-positive because (1) more inference throughput tends to increase top-of-rack connectivity requirements, (2) distributing inference across more racks and sites increases short-reach links per unit of delivered service, and (3) PCIe-attached accelerator architectures tend to require robust signal conditioning as systems move to PCIe 6.x and beyond. Groq workshop materials explicitly reference GroqCard and GroqNode form factors, reinforcing that PCIe-attached deployment has been central to Groq’s current packaging strategy. The main countervailing risk is that Groq’s deterministic chip-to-chip fabric could be implemented primarily through backplanes and direct board-level connectivity that reduces the need for merchant AECs inside the box; in that case, incremental AEC demand would concentrate more in rack-to-switch and node-to-fabric links rather than within-chassis chip fabrics. Astera Labs implications are connectivity-architecture sensitive and, on balance, skew positive if NVIDIA increases heterogeneity and disaggregation in AI systems. NVIDIA has publicly positioned NVLink Fusion as a pathway for partners to build semi-custom AI infrastructure and has explicitly identified Astera Labs as a partner in that ecosystem, with Astera describing NVLink-related solutions expanding its connectivity platform across PCIe, CXL, and Ethernet plus fleet observability software. A Groq acquisition increases the probability that NVIDIA offers a broader menu of accelerators (training GPUs, inference-focused ASICs) and therefore increases the importance of scalable, high-reliability connectivity, retiming, switching, and telemetry across mixed topologies. If Groq silicon remains PCIe-attached in many deployments, PCIe 6.x retimers/switches and active cable modules become more central, aligning with Astera’s core portfolio. If NVIDIA instead integrates Groq concepts into scale-up fabrics (NVLink-like domains) or uses Groq to expand into inference “appliances” that must be rapidly deployed in colocation environments, the need for standard-compliant, serviceable connectivity with strong RAS/telemetry increases, again aligning with Astera’s positioning. Power equipment and cooling implications for Vertiv and adjacent suppliers should be viewed through the lens of rack power density, cooling modality (air vs liquid), and site deployment model (hyperscale campuses vs distributed colocation/enterprise). Groq claims its LPU and rack designs are “air-cooled by design” and require no complex cooling and power infrastructure, and third-party reporting has described Groq’s approach as relying on parallelism across many lower-power units rather than extreme per-chip performance. If NVIDIA scales Groq as a mainstream inference platform, the mix of data center cooling spend could shift modestly away from the highest-density liquid-cooled racks toward more air-cooled or hybrid deployments, particularly for inference pods placed in existing facilities that cannot easily retrofit for very high rack heat flux. That would be a mix headwind for suppliers most levered exclusively to high-end liquid cooling attachments per rack, but it is not necessarily a volume headwind for Vertiv given the company’s broad exposure to both power and cooling infrastructure and the likelihood that total AI deployment locations expand. Vertiv’s own industry commentary emphasizes that AI racks require higher power-density UPS, batteries, power distribution equipment, and switchgear capable of handling rapid load transients, and that hybrid cooling systems will evolve across deployment environments. Those statements align with a world where inference growth increases the count of powered racks and raises the operational complexity of power delivery even if per-rack density is lower than the most extreme training clusters. The most material infrastructure impact may occur outside the rack and upstream of the data hall: grid interconnects, substations, transformers, switchgear, generators, and utility-scale generation additions. Recent regulatory actions in the U.S. highlight that projected data center demand is already driving large planned increases in electricity generation capacity, underscoring that power availability is a binding constraint. In that context, an inference architecture that lowers joules per token could reduce the power required per unit of inference delivered, but it can also accelerate demand by lowering cost and improving latency, increasing the total volume of inference served (a classic rebound effect). The net outcome is likely continued, elevated demand for power infrastructure even if efficiency improves, with the key swing factor being whether AI capex remains on a multi-year growth trajectory or enters a digestion phase. Other data center infrastructure implications include server/ODM mix, facility design standardization, and networking architecture choices. If NVIDIA positions Groq-based inference as a broadly distributable “standard server + accelerator” solution rather than as an integrated, liquid-cooled rack like GB200 NVL72, spend could shift toward more conventional air-cooled server designs, higher unit volumes of mainstream racks, and faster deployment in colocation footprints, increasing demand for modular power rooms, busways, and rapidly deployable cooling solutions. If NVIDIA instead integrates Groq into its “AI factory” paradigm, the primary effect is likely acceleration of dense back-end fabric build-outs and a faster push toward photonics switching, increasing demand for fiber plant, connectors, and integrated optics supply chains while potentially compressing the lifecycle of transitional architectures based on pluggable optics and mid-reach copper. NVIDIA’s stated roadmap toward co-packaged optics and silicon photonics switches is already oriented toward scaling to very large GPU counts; adding a high-end inference ASIC increases the strategic importance of power-efficient, low-latency fabrics because inference economics become increasingly sensitive to network overhead as compute cost declines. Across the covered segments, the most defensible base case is limited near-term dislocation and a medium-term increase in uncertainty around memory intensity per unit of inference growth. HBM faces the clearest relative risk from an HBM-less inference platform, but supply tightness and GPU training roadmaps reduce the probability of an absolute demand shock over the next 12–24 months. Optical, AEC/copper, and power/cooling are more likely to remain volume-supported because they scale with endpoint count, deployment fragmentation, and total data center footprint, and those tend to rise when inference becomes cheaper and more widely deployed. The highest-conviction second-order effect is a shift in infrastructure mix: incrementally more distributed inference deployments (favoring colocation power/cooling standardization, DCI optics, and serviceable short-reach interconnect) and a gradual migration from pluggable optics toward integrated photonics in back-end fabrics (favoring suppliers positioned in the CPO ecosystem).

$MU $SNDK $LITE $VRT NVIDIA and Groq: 2nd and 3rd Order Strategic Infrastructure Effects and Market Implications Public reporting indicates NVIDIA has agreed to acquire Groq for approximately $20,000,000,000 in cash, while excluding Groq’s nascent cloud business from the transaction perimeter. The reported carve-out materially constrains the immediate, direct linkage from the acquisition to incremental, NVIDIA-controlled data center capacity build-out because GroqCloud appears to be the principal channel through which Groq hardware is currently monetized at scale as a service. The infrastructure-market implications therefore depend primarily on post-close product strategy: whether NVIDIA (1) commercializes Groq silicon as a distinct inference product line and drives broad deployment through OEM/ODM channels and partners, (2) uses the acquisition mainly to absorb IP and talent while de-emphasizing standalone Groq hardware volumes, or (3) uses Groq technology to reshape NVIDIA’s own inference systems and networking roadmaps. The dominant transmission mechanism into memory, networking, and facility infrastructure markets is the degree to which NVIDIA shifts incremental inference deployments away from GPU architectures that are tightly coupled to external high-bandwidth memory (HBM) and toward Groq’s current architecture, which emphasizes large on-chip SRAM, deterministic compiler-scheduled execution, and direct chip-to-chip connectivity. Independent and company-published materials describe Groq’s current-generation approach as having no external memory, keeping weights and KV cache on-chip during processing, and requiring model sharding across multiple chips due to limited on-chip SRAM per device. That architectural choice is directionally HBM-negative on a per-accelerator basis and ambiguous for DRAM, NAND, networking, power, and cooling on a per-token basis because the design can reduce memory wall losses and tail-latency overhead while potentially increasing the number of chips and interconnect endpoints required to serve large models and long-context workloads. HBM implications are the most mechanically straightforward but should be framed as second-derivative rather than absolute. If Groq-class inference silicon meaningfully displaces NVIDIA GPU-based inference deployments, incremental HBM bit demand tied to inference growth could be reduced relative to a GPU-only baseline because Groq’s current approach does not appear to attach HBM stacks to each accelerator. However, current market structure suggests HBM remains supply-constrained and is being pulled by multiple vectors including continued GPU training scale and high-capacity inference configurations, with leading suppliers signaling tight conditions extending beyond 2026. In that environment, reduced inference-driven HBM intensity could primarily reallocate scarce HBM supply toward higher-end training and premium inference GPUs rather than creating an outright volume collapse, preserving high utilization of HBM capacity while potentially affecting the slope of pricing power and capacity expansion urgency over a multi-year horizon. The key downside scenario for the HBM complex would be a durable architectural bifurcation where “good-enough” inference shifts disproportionately to HBM-less ASICs across a broad swath of deployments (latency-sensitive, batch-1, cost-per-token optimized), while training remains GPU-HBM dominated; such a split would reduce the portion of future inference compute that naturally monetizes through HBM content and could compress the incremental HBM-per-AI-dollar ratio. The key upside/neutral scenario for HBM is that the supply chain remains fully allocated regardless, with NVIDIA using any “freed” HBM to ship more high-end GPUs into training and long-context inference, especially as roadmaps increase HBM per GPU, sustaining robust aggregate bit demand even if inference becomes more heterogeneous. Conventional DRAM implications split into 2 channels: (1) DRAM wafer capacity diversion into HBM and (2) DDR content per server in AI clusters. Supplier commentary indicates that AI-driven memory demand is supporting elevated DRAM markets more broadly, and HBM production is resource-intensive versus conventional DRAM, tightening supply for DDR products in parallel. A meaningful NVIDIA pivot to an inference architecture that reduces HBM dependence could, at the margin, ease the most acute HBM-driven bottlenecks and allow memory manufacturers more flexibility in balancing DRAM mix, which could be modestly DDR-positive on the supply side (less crowding-out) even if it is DDR-neutral or slightly negative on the demand side (if per-node CPU/DDR requirements decline due to more efficient accelerator utilization). The dominant practical outcome is likely that DDR demand remains supported by broad AI server proliferation and increasing memory footprints at the system level (CPUs, networking stacks, caching layers, retrieval-augmented pipelines), while HBM remains the premium profit pool; therefore, any HBM displacement that increases total server volumes could indirectly keep DDR demand resilient even if DDR per accelerator is not rising materially. NAND flash implications are comparatively indirect and volume-driven rather than architecture-driven. Inference clusters require SSD capacity for model storage, container images, logging, and increasingly for fast local retrieval indices and embedding stores, but the storage footprint per unit of compute is typically smaller than in training pipelines that stage large datasets and checkpoints. If NVIDIA uses Groq to lower inference cost and latency enough to expand the total number of inference deployment locations (regional colocation, enterprise on-prem, sovereign footprints), aggregate SSD attach could rise through geographic fragmentation and replication of model artifacts across more sites, even if per-site storage is modest. The NAND effect is therefore likely to be demand-broadening and mix-positive (datacenter SSDs) but not a primary swing factor versus the macro AI capex cycle and consumer/device cycles. Hard disk drive (HDD) markets should see negligible direct sensitivity because nearline HDD demand is driven by bulk storage and cloud archiving economics, while inference acceleration choices primarily reshape compute and network layers; any HDD benefit would be a tertiary function of overall data center square footage expansion rather than a direct consequence of Groq silicon displacing GPUs. Optical networking implications require separating (1) intra-cluster back-end fabrics that connect accelerators and (2) front-end / data center interconnect (DCI) that connects sites and regions. Groq’s own positioning and third-party reporting suggest scaling beyond a single node or rack relies on high-bandwidth fabrics and, in some described configurations, optical interconnect scaling across hundreds of chips. If NVIDIA commercializes Groq at scale, 2 offsetting forces emerge: lower cost-per-token and improved latency could expand inference throughput and drive more east-west traffic, increasing demand for high-speed switching and optics; conversely, if Groq delivers materially higher utilization and tokens per unit of network bandwidth for certain workloads, the network required per served token could decline. Public NVIDIA materials already indicate an aggressive photonics roadmap aimed at scaling AI factories, including co-packaged optics (CPO) switches and explicit collaboration with Coherent and Lumentum in the silicon photonics supply chain. That linkage is important because it suggests that, independent of Groq, NVIDIA is already pushing optics integration deeper into the switch package to reduce power and increase resiliency; Groq increases the strategic incentive to reduce network power and latency if inference becomes even more distributed and latency-sensitive. For Lumentum and Coherent specifically, the net implication is less about “more optics versus fewer optics” and more about a shift in optics form factor and value capture. Co-packaged optics can reduce reliance on pluggable transceivers in some switch architectures while increasing demand for integrated photonic engines, lasers, fiber attach, packaging processes, and component-level supply. NVIDIA’s own announcements explicitly position Coherent and Lumentum as collaborators in creating the integrated silicon/optics process and supply chain for photonics switches. If Groq accelerates the transition to very large-scale fabrics (more endpoints, higher port speeds, tighter power envelopes), that tends to pull forward CPO adoption and amplifies demand for the underlying photonics components even if the conventional pluggable module TAM is structurally pressured over time. If Groq instead pushes inference toward smaller, more localized pods (closer to users, more regional colocation), that can be optics-positive for DCI and metro connectivity because more sites must be interconnected at high bandwidth with low latency, favoring coherent optics and high-speed interconnect between facilities. The principal risk for optics suppliers is timing and margin structure: a faster move to NVIDIA-driven integrated photonics could concentrate bargaining power and compress margins for commoditized transceiver modules while favoring suppliers with differentiated lasers, integration capability, and qualification depth in NVIDIA’s CPO ecosystem. AEC and copper interconnect implications hinge on whether Groq deployment increases the density of short-reach links inside racks and rows. High-speed copper remains structurally advantaged at very short distances on cost, power, and serviceability, but reaches become constrained as lane speeds and aggregate bandwidth rise, creating a role for active electrical cables (AECs), retimers, and signal-conditioning silicon. Credo explicitly positions its AEC products as enabling reliable lossless 800G connectivity for AI clusters, and the company has highlighted participation at NVIDIA GTC with content focused on extending PCIe/CXL using AECs, indicating relevance to next-generation system topologies that require longer reach and higher signal integrity than passive copper can deliver. If NVIDIA turns Groq into a widely deployed inference card or chassis product, the likely near-term effect is AEC-positive because (1) more inference throughput tends to increase top-of-rack connectivity requirements, (2) distributing inference across more racks and sites increases short-reach links per unit of delivered service, and (3) PCIe-attached accelerator architectures tend to require robust signal conditioning as systems move to PCIe 6.x and beyond. Groq workshop materials explicitly reference GroqCard and GroqNode form factors, reinforcing that PCIe-attached deployment has been central to Groq’s current packaging strategy. The main countervailing risk is that Groq’s deterministic chip-to-chip fabric could be implemented primarily through backplanes and direct board-level connectivity that reduces the need for merchant AECs inside the box; in that case, incremental AEC demand would concentrate more in rack-to-switch and node-to-fabric links rather than within-chassis chip fabrics. Astera Labs implications are connectivity-architecture sensitive and, on balance, skew positive if NVIDIA increases heterogeneity and disaggregation in AI systems. NVIDIA has publicly positioned NVLink Fusion as a pathway for partners to build semi-custom AI infrastructure and has explicitly identified Astera Labs as a partner in that ecosystem, with Astera describing NVLink-related solutions expanding its connectivity platform across PCIe, CXL, and Ethernet plus fleet observability software. A Groq acquisition increases the probability that NVIDIA offers a broader menu of accelerators (training GPUs, inference-focused ASICs) and therefore increases the importance of scalable, high-reliability connectivity, retiming, switching, and telemetry across mixed topologies. If Groq silicon remains PCIe-attached in many deployments, PCIe 6.x retimers/switches and active cable modules become more central, aligning with Astera’s core portfolio. If NVIDIA instead integrates Groq concepts into scale-up fabrics (NVLink-like domains) or uses Groq to expand into inference “appliances” that must be rapidly deployed in colocation environments, the need for standard-compliant, serviceable connectivity with strong RAS/telemetry increases, again aligning with Astera’s positioning. Power equipment and cooling implications for Vertiv and adjacent suppliers should be viewed through the lens of rack power density, cooling modality (air vs liquid), and site deployment model (hyperscale campuses vs distributed colocation/enterprise). Groq claims its LPU and rack designs are “air-cooled by design” and require no complex cooling and power infrastructure, and third-party reporting has described Groq’s approach as relying on parallelism across many lower-power units rather than extreme per-chip performance. If NVIDIA scales Groq as a mainstream inference platform, the mix of data center cooling spend could shift modestly away from the highest-density liquid-cooled racks toward more air-cooled or hybrid deployments, particularly for inference pods placed in existing facilities that cannot easily retrofit for very high rack heat flux. That would be a mix headwind for suppliers most levered exclusively to high-end liquid cooling attachments per rack, but it is not necessarily a volume headwind for Vertiv given the company’s broad exposure to both power and cooling infrastructure and the likelihood that total AI deployment locations expand. Vertiv’s own industry commentary emphasizes that AI racks require higher power-density UPS, batteries, power distribution equipment, and switchgear capable of handling rapid load transients, and that hybrid cooling systems will evolve across deployment environments. Those statements align with a world where inference growth increases the count of powered racks and raises the operational complexity of power delivery even if per-rack density is lower than the most extreme training clusters. The most material infrastructure impact may occur outside the rack and upstream of the data hall: grid interconnects, substations, transformers, switchgear, generators, and utility-scale generation additions. Recent regulatory actions in the U.S. highlight that projected data center demand is already driving large planned increases in electricity generation capacity, underscoring that power availability is a binding constraint. In that context, an inference architecture that lowers joules per token could reduce the power required per unit of inference delivered, but it can also accelerate demand by lowering cost and improving latency, increasing the total volume of inference served (a classic rebound effect). The net outcome is likely continued, elevated demand for power infrastructure even if efficiency improves, with the key swing factor being whether AI capex remains on a multi-year growth trajectory or enters a digestion phase. Other data center infrastructure implications include server/ODM mix, facility design standardization, and networking architecture choices. If NVIDIA positions Groq-based inference as a broadly distributable “standard server + accelerator” solution rather than as an integrated, liquid-cooled rack like GB200 NVL72, spend could shift toward more conventional air-cooled server designs, higher unit volumes of mainstream racks, and faster deployment in colocation footprints, increasing demand for modular power rooms, busways, and rapidly deployable cooling solutions. If NVIDIA instead integrates Groq into its “AI factory” paradigm, the primary effect is likely acceleration of dense back-end fabric build-outs and a faster push toward photonics switching, increasing demand for fiber plant, connectors, and integrated optics supply chains while potentially compressing the lifecycle of transitional architectures based on pluggable optics and mid-reach copper. NVIDIA’s stated roadmap toward co-packaged optics and silicon photonics switches is already oriented toward scaling to very large GPU counts; adding a high-end inference ASIC increases the strategic importance of power-efficient, low-latency fabrics because inference economics become increasingly sensitive to network overhead as compute cost declines. Across the covered segments, the most defensible base case is limited near-term dislocation and a medium-term increase in uncertainty around memory intensity per unit of inference growth. HBM faces the clearest relative risk from an HBM-less inference platform, but supply tightness and GPU training roadmaps reduce the probability of an absolute demand shock over the next 12–24 months. Optical, AEC/copper, and power/cooling are more likely to remain volume-supported because they scale with endpoint count, deployment fragmentation, and total data center footprint, and those tend to rise when inference becomes cheaper and more widely deployed. The highest-conviction second-order effect is a shift in infrastructure mix: incrementally more distributed inference deployments (favoring colocation power/cooling standardization, DCI optics, and serviceable short-reach interconnect) and a gradual migration from pluggable optics toward integrated photonics in back-end fabrics (favoring suppliers positioned in the CPO ecosystem).

TheValueist

76,267 görüntüleme • 9 ay önce

$GRAB Map is The New Google Maps(B2B)🧵 Here is your Free.99 analysis on GrabMap, for those that selling courses for $50-$500/m, if you are using my $GRAB and other analyses, I don't ask for much, at least give me some credit/cite. And yes 99.999% of my posts are Free.99. If you want to support my work, slap the like/repost, as I don't choose to write "Grab or any Ticker is going to x10 x100-x1000" kind of threads or "mark my words" to please the X Algo. Consider Subscribe($0.33/day) if you want to support my work further and get more in-depth analyses! TLDR: GrabMap could generate $7B-$15B a year alone for Grab B2B segment. That is why you are seeing Anthony Tan is mad excited abt this massive opportunity. And it also significantly boost GrabAds long term globally. This precisely proved my point that, Anthony is going to expand to 5B people and we are only 14% thesis realized right now. Grab doesn't have to be just Ride-share/Delivery when expanding! Grab , Southeast Asia's leading AI SuperApp for ride-hailing, food delivery, financial services,Tourism, Dine-Out and more, has developed its proprietary mapping platform, GrabMaps, a massive B2B revenue potential over the next long term, not just in Singapore, Indonesia, Malaysia, Thailand, Philippines, Vietnam, Cambodia, and Myanmar but expanding beyond SEA markets/Customers. 1. GrabMaps: A Strategic Asset GrabMaps is not merely a technological tool but a critical component of Grab's ecosystem, powering its ride-hailing, food delivery, and financial services. Developed in-house, GrabMaps leverages data collected from Grab's vast network of driver-partners across eight SEA countries. This data-driven approach ensures hyper-local customization, addressing the unique challenges of SEA's urban environments, such as narrow alleys, informal roads, and rapid infrastructure changes. The recent announcement of KartaCam2, an upgraded street-level imaging device, marks a significant technological advancement. KartaCam2 enhances data collection by providing higher quality images and more precise location data, which are crucial for maintaining the accuracy and freshness of maps. This breakthrough is part of Grab's broader 2025 AI push, including integrations with OpenAI 's GPT-4o for vision-based mapping and the establishment of an AI Centre of Excellence. These innovations position GrabMaps as a formidable competitor to Google Maps, especially in regions where localized data is paramount. 2. Revenue implications long term The expansion of GrabMaps into B2B services opens up new revenue streams, which could significantly impact Grab's financial performance over the long term. But GrabMap is a brandnew B2B product, and GoogleMap generates around $13-$20B globally. A. Market Opportunity in Southeast Asia ~The SEA market presents a substantial opportunity for GrabMaps. The foodservice market alone is projected to grow from $223.8 billion in 2025 to $416.3 billion by 2030, indicating a robust demand for services that enhance operational efficiencies. Businesses in logistics, e-commerce, and urban planning could benefit from GrabMaps' precise mapping and navigation capabilities, potentially generating revenue through licensing fees, subscription models, and advertising. ~Grab's existing user base of over 46 million monthly transacting users provides a strong foundation for cross-selling B2B solutions, thereby increasing revenue without significant additional marketing costs. B. Competitive Advantage of a Future $500B MC AI SuperApp over Google Map Google Maps, while dominant, may not be as finely tuned for SEA's unique challenges. GrabMaps' hyper-local data and AI-driven enhancements offer a competitive edge, attracting businesses that require accurate and cost-effective mapping solutions. Revenue from B2B services could include: Licensing Fees: Enterprises can license GrabMaps' APIs and SDKs to integrate mapping functionalities into their operations. Subscription Models: Continuous updates and premium features could be offered on a subscription basis. Advertising Revenue: GrabAds, which leverages mapping data, could generate additional income through targeted advertising. C. Global Expansion is Inevitable ~The partnership with Tino in Mongolia is a strategic move to scale GrabMaps internationally. This marks Grab's first major mapping partnership outside SEA, indicating potential for revenue growth in other regions where Google Maps' dominance is less entrenched or where local data needs are acute. ~The use of IoT devices like KartaCam2 and KartaDashCam for real-time data collection could further enhance GrabMaps' value proposition, potentially increasing revenue through premium service offerings in new markets. D. Synergies w/ other businesses Grab's ecosystem approach allows for synergies between GrabMaps and other services like GrabPay, GrabFood, and GrabTransport. For example, businesses using GrabMaps for logistics could also adopt GrabPay for transactions, creating a revenue multiplier effect. 3. Google Map Revenue in Asia ~Total Revenue in Asia-Pacific (2018): Google APAC, based in Singapore, reported $20.24 billion out of the total $21.37 billion revenue in the Asia-Pacific region. This indicates that a significant portion of Google's revenue in Asia is attributed to Singapore, likely due to its role as a hub for Google’s operations. ~Advertising Revenue: In 2018, Google APAC generated $15.8 billion from advertising alone, compared to $4.4 billion from other activities like Google Play. Advertising on Google properties, including Google Maps, is a major revenue driver. ~Market Share in Search Marketing: Google Maps holds a 62.34% market share in the search marketing category, competing with tools like Wix (26.54%) and Google Ads (4.14%). This dominance suggests that a considerable portion of Google’s advertising revenue in Asia is linked to mapping services. For the full fiscal year 2024, Alphabet (Google's parent company) generated $56.82 billion in revenue from the Asia-Pacific (APAC) region. This represented approximately 16.24% of the company's total revenue for the year. If we take a conservative estimate at 25% of $56.82B of Google's total advertising revenue in Asia is related to mapping services= $14.2B. => If GrabMaps secures even 50% of this market share in SEA, it could generate around $7B annually from this segment alone. GrabMap is 4x lower error rate, 10x lower latency, 75% fewer mapping mistakes, and much cheaper than GoogleMap. With OpenAI GPT-4o fine-tuning, GrabMaps hit 80% accuracy for speed limits and lanes13-20% above prior levels excelling in occlusions ( rainy monsoons) where Google relies more on satellite data. Now do you understand why Google and HSBC are clapping $GRAB on search and downgrade? Yes, because GrabMap is a massive threat and Grab Anthony Tan refused to buy $goto since 2020. Conclusion: Grab's expansion of GrabMaps into B2B services represents a strategic move to challenge Google Maps' dominance in Asia, particularly in SEA and future expansion. The revenue implications are substantial, with potential gains from licensing fees, subscription models, advertising, and international expansions. While Google Maps generates billions in revenue, primarily through advertising, GrabMaps' localized and AI-enhanced approach could carve out a significant niche, especially in regions where precise, real-time mapping data is critical. The success of this strategy will depend on Grab's ability to scale internationally, maintain technological superiority, and effectively monetize its B2B offerings. However, the opportunity is clear, and Grab's ecosystem approach positions it well to capitalize on the growing demand for advanced mapping solutions in a rapidly digitalizing world. This move not only enhances Grab's revenue potential but also solidifies its role as a key player in the global tech landscape. Not Financial Advice! Source: Grab Dot Com.

$GRAB Map is The New Google Maps(B2B)🧵 Here is your Free.99 analysis on GrabMap, for those that selling courses for $50-$500/m, if you are using my $GRAB and other analyses, I don't ask for much, at least give me some credit/cite. And yes 99.999% of my posts are Free.99. If you want to support my work, slap the like/repost, as I don't choose to write "Grab or any Ticker is going to x10 x100-x1000" kind of threads or "mark my words" to please the X Algo. Consider Subscribe($0.33/day) if you want to support my work further and get more in-depth analyses! TLDR: GrabMap could generate $7B-$15B a year alone for Grab B2B segment. That is why you are seeing Anthony Tan is mad excited abt this massive opportunity. And it also significantly boost GrabAds long term globally. This precisely proved my point that, Anthony is going to expand to 5B people and we are only 14% thesis realized right now. Grab doesn't have to be just Ride-share/Delivery when expanding! Grab , Southeast Asia's leading AI SuperApp for ride-hailing, food delivery, financial services,Tourism, Dine-Out and more, has developed its proprietary mapping platform, GrabMaps, a massive B2B revenue potential over the next long term, not just in Singapore, Indonesia, Malaysia, Thailand, Philippines, Vietnam, Cambodia, and Myanmar but expanding beyond SEA markets/Customers. 1. GrabMaps: A Strategic Asset GrabMaps is not merely a technological tool but a critical component of Grab's ecosystem, powering its ride-hailing, food delivery, and financial services. Developed in-house, GrabMaps leverages data collected from Grab's vast network of driver-partners across eight SEA countries. This data-driven approach ensures hyper-local customization, addressing the unique challenges of SEA's urban environments, such as narrow alleys, informal roads, and rapid infrastructure changes. The recent announcement of KartaCam2, an upgraded street-level imaging device, marks a significant technological advancement. KartaCam2 enhances data collection by providing higher quality images and more precise location data, which are crucial for maintaining the accuracy and freshness of maps. This breakthrough is part of Grab's broader 2025 AI push, including integrations with OpenAI 's GPT-4o for vision-based mapping and the establishment of an AI Centre of Excellence. These innovations position GrabMaps as a formidable competitor to Google Maps, especially in regions where localized data is paramount. 2. Revenue implications long term The expansion of GrabMaps into B2B services opens up new revenue streams, which could significantly impact Grab's financial performance over the long term. But GrabMap is a brandnew B2B product, and GoogleMap generates around $13-$20B globally. A. Market Opportunity in Southeast Asia ~The SEA market presents a substantial opportunity for GrabMaps. The foodservice market alone is projected to grow from $223.8 billion in 2025 to $416.3 billion by 2030, indicating a robust demand for services that enhance operational efficiencies. Businesses in logistics, e-commerce, and urban planning could benefit from GrabMaps' precise mapping and navigation capabilities, potentially generating revenue through licensing fees, subscription models, and advertising. ~Grab's existing user base of over 46 million monthly transacting users provides a strong foundation for cross-selling B2B solutions, thereby increasing revenue without significant additional marketing costs. B. Competitive Advantage of a Future $500B MC AI SuperApp over Google Map Google Maps, while dominant, may not be as finely tuned for SEA's unique challenges. GrabMaps' hyper-local data and AI-driven enhancements offer a competitive edge, attracting businesses that require accurate and cost-effective mapping solutions. Revenue from B2B services could include: Licensing Fees: Enterprises can license GrabMaps' APIs and SDKs to integrate mapping functionalities into their operations. Subscription Models: Continuous updates and premium features could be offered on a subscription basis. Advertising Revenue: GrabAds, which leverages mapping data, could generate additional income through targeted advertising. C. Global Expansion is Inevitable ~The partnership with Tino in Mongolia is a strategic move to scale GrabMaps internationally. This marks Grab's first major mapping partnership outside SEA, indicating potential for revenue growth in other regions where Google Maps' dominance is less entrenched or where local data needs are acute. ~The use of IoT devices like KartaCam2 and KartaDashCam for real-time data collection could further enhance GrabMaps' value proposition, potentially increasing revenue through premium service offerings in new markets. D. Synergies w/ other businesses Grab's ecosystem approach allows for synergies between GrabMaps and other services like GrabPay, GrabFood, and GrabTransport. For example, businesses using GrabMaps for logistics could also adopt GrabPay for transactions, creating a revenue multiplier effect. 3. Google Map Revenue in Asia ~Total Revenue in Asia-Pacific (2018): Google APAC, based in Singapore, reported $20.24 billion out of the total $21.37 billion revenue in the Asia-Pacific region. This indicates that a significant portion of Google's revenue in Asia is attributed to Singapore, likely due to its role as a hub for Google’s operations. ~Advertising Revenue: In 2018, Google APAC generated $15.8 billion from advertising alone, compared to $4.4 billion from other activities like Google Play. Advertising on Google properties, including Google Maps, is a major revenue driver. ~Market Share in Search Marketing: Google Maps holds a 62.34% market share in the search marketing category, competing with tools like Wix (26.54%) and Google Ads (4.14%). This dominance suggests that a considerable portion of Google’s advertising revenue in Asia is linked to mapping services. For the full fiscal year 2024, Alphabet (Google's parent company) generated $56.82 billion in revenue from the Asia-Pacific (APAC) region. This represented approximately 16.24% of the company's total revenue for the year. If we take a conservative estimate at 25% of $56.82B of Google's total advertising revenue in Asia is related to mapping services= $14.2B. => If GrabMaps secures even 50% of this market share in SEA, it could generate around $7B annually from this segment alone. GrabMap is 4x lower error rate, 10x lower latency, 75% fewer mapping mistakes, and much cheaper than GoogleMap. With OpenAI GPT-4o fine-tuning, GrabMaps hit 80% accuracy for speed limits and lanes13-20% above prior levels excelling in occlusions ( rainy monsoons) where Google relies more on satellite data. Now do you understand why Google and HSBC are clapping $GRAB on search and downgrade? Yes, because GrabMap is a massive threat and Grab Anthony Tan refused to buy $goto since 2020. Conclusion: Grab's expansion of GrabMaps into B2B services represents a strategic move to challenge Google Maps' dominance in Asia, particularly in SEA and future expansion. The revenue implications are substantial, with potential gains from licensing fees, subscription models, advertising, and international expansions. While Google Maps generates billions in revenue, primarily through advertising, GrabMaps' localized and AI-enhanced approach could carve out a significant niche, especially in regions where precise, real-time mapping data is critical. The success of this strategy will depend on Grab's ability to scale internationally, maintain technological superiority, and effectively monetize its B2B offerings. However, the opportunity is clear, and Grab's ecosystem approach positions it well to capitalize on the growing demand for advanced mapping solutions in a rapidly digitalizing world. This move not only enhances Grab's revenue potential but also solidifies its role as a key player in the global tech landscape. Not Financial Advice! Source: Grab Dot Com.

Mike

120,532 görüntüleme • 11 ay önce

$AMD| The FOMO to buy AMD Chips is NOW 🧵 Not Financial Advice! DYOR! Research Purpose Only! The Inference Queen is the biggest winner in Agentic AI where all other CPUs are struggling to compete with a 2yr old EPYC Turin and EPYC Venice is in mass production phase. AMD stresses deployability today on standard x86 platforms (no proprietary architectures required), full software compatibility, and open standards. This positions Venice + Helios as a practical, high-density alternative to competing solutions while underscoring that agentic AI shifts the balance toward CPU-rich racks alongside GPUs, and most importantly, lowering the cost of token to accelerate adoption and innovation. Context: The Wall Street Journal yesterday came out with an article that OpenAI is condiering drasstically lowering the token prices to win more customers from Anthropic. The narrative "they" are trying to exacerbate the current AI selloff won't last long. This is a fundamental misunderstanding of what is going on, or what I already discussed for months and years. Followers and Subscribers already knew this for years, that this day would come, where token cost will bcome the central discussion among enterprises as there is no such thing as unlimited budget or Tokenmaxxing when they use $NVDA chips or In-house Hyperscalers chips. I will link various threads if you are interested in understanding the full picture from supply chain to recent TSMC Rapid 2nm expansion up to 12 Fabs total by 2027/2028. Hyperscalers and AI natives effectively have no choice but to buy more AMD system for Agentic AI as leadership in economical, power-aware, high-volume internal + agentic use. However, due to supply constraints where Supply is far behind Demand, this makes multi-vendor reality along with in-house chips drive faster industry progress, lower overall costs, and better sustainability. NVIDIA’s Vera Rubin cannot compete with a 2 years old EPYC Turin, but AMD under Dr. Lisa Su has engineered the lowest cost-per-million-tokens, highly competitive energy-efficient solutions, and superior CPU orchestration for agentic AI at scale with Helios. Dr. Su has championed this shift since at least 2023, foreseeing the rise of agentic workflows that demand far more orchestration, parallel agents, and balanced compute well before the industry fully embraced it. Her long-term vision of AI moving from simple prompts to always on, multi-agent systems has driven AMD’s investments in high-core EPYC CPUs and integrated rack-scale solutions, perfectly positioning the company for today’s realities. The OpenAI-AMD 1GW Helios deployment (starting H2 2026) represents a pivotal vertical integration move that directly supercharges the inference economics. This isn't incremental; it's a structural shift toward ownership of massive, optimized rack-scale capacity, enabling the lowest token costs and triggering the enterprise adoption flywheel. We need to be honest, $AMD is the only company that made a big bet on Inference since the day Chatgpt became sensational where $NVDA and others were betting big on Training. At the end of the day, Token bill from Anthropic has to obey economics. Meaning the bills rise, companies have to get more out of it to justify the cost. It cannot be an unlimited inference budget, and it has to show up on efficiency, profitability and operating leverage. 1. Tokenomics After you understand this, you will understand why Citi cited Anthropic is likely to sign a deal with $AMD along with Hyperscalers, AI Labs, Sovereign AI like Softbank 5GW in France and many other countries. However, OpenAI and $META are now wanting faster deployment, and they are AMD shareholders now, they have prioritized allocation. Anthropic and Hyperscalers just cannot compete when Helios Rack lower token cost to$0.0003–$0.0005 per million tokens at GW scale. Cost to build 1GW data center 1GW Helios Rack full build is estimated $30-$35B 1GW Rubin Rack full build is estimated $45-$55B Inference (Cost per Million Tokens) ~$NVDA B200 / HGX: ~$0.02–$0.08 on optimized workloads (FP4/MXFP4, speculative decoding). Significant improvement over Hopper but still premium-priced. GB200 NVL72 rack-scale: $0.05–$0.25+ ~$AMD Helios Racks: $0.0003-$0.0005 per M tokens, dramatically lower than NVIDIA equivalents in owned infra. MI355X node-level: Up to 40% more tokens per dollar vs. competing solutions ( B200), driven by higher memory capacity (up to 288GB+ HBM), strong bandwidth, and lower acquisition costs. Training ~$NVDA Rubin Rack is estimated $0.7-$1.2/M Tokens ~$AMD Helios Rack is estimated $0.65-$1.0/M Tokens Now, OpenAI, META and Hyperscalers can lower Inference cost even further with $AMD EPYC Venice "dense rack" or Agentic AI Rack. AMD published a detailed technical blog emphasizing that the future of agentic AI autonomous, multi-step AI systems requiring heavy orchestration, databases, caching, APIs, and control planes demands massive CPU-dense rack-scale infrastructure, not just GPUs. The catalyst prominently positions their upcoming 6th Gen EPYC "Venice" processors as the key enabler for next-generation dense racks, delivering leadership throughput under real-world power, cooling, and density constraints. ~EPYC Venice (Zen 6 architecture, up to 256 cores / 512 threads per socket) is projected to deliver exceptional rack-level performance. In AMD’s modeled 100 kW rack comparisons, Venice-powered systems are expected to achieve ~3.30x the throughput of NVIDIA’s Vera (88-core Olympus) baseline across a broad mix of agentic-supporting workloads. ~This builds on current-generation 5th Gen EPYC "Turin" (up to 192 cores), which already delivers ~2.37x rack throughput vs. Vera and ~1.6x vs. Intel’s Xeon 6980P (128 cores). ~ Liquid-cooled Turin deployments already support >27,000 CPU cores per rack today. Venice is architected to push this beyond 36,000 cores in the same rack class, dramatically increasing concurrent agent capacity and overall infrastructure efficiency. 2. Ownership vs renting compute from Hyperscalers matter to OpenAI and only owning $AMD chips can meaningfully lower token cost for enterprises. ~Eliminates cloud overhead: No provider margins, utilization buffers, or egress fees. Direct control over power contracts, cooling, scheduling, and orchestration at dedicated facilities. ~Helios optimizations at GW scale: Rack-level density (1.4+ exaFLOPS FP8 per rack), high HBM4 bandwidth, EPYC orchestration for agentic workloads, and superior TCO/TDP. AMD's long-standing focus on tokens per dollar/watt shines here 20-40%+ efficiency edges in inference-heavy scenarios. ~At 1GW+ optimized deployment, inference hits $0.0003–$0.0005 per million tokens (community/analyst models tied to Helios metrics). This is dramatically lower than typical rented/cloud equivalents, especially for high-volume output tokens in agentic flows. High token bills today, enterprises running heavy agentic/coding/analysis workloads can face $50-100M+/month at current API rates (flagship models $5-30+/M output, scaled to massive volumes). Post-Helios compression, same volume will drop to $10-15M/month (or better) via lower underlying costs passed through as pricing flexibility, volume tiers, caching, or batch discounts. ROI thresholds collapse. More companies greenlight pilots → production → massive scaling. Agentic AI (autonomous workflows) multiplies token demand exponentially, but affordability removes the friction. OpenAI gains flexibility, Unlike more cloud-dependent rivals (Anthropic), they can lower effective pricing, offer aggressive enterprise bundles, or absorb volume without margin destruction directly tackling "high token bill" complaints while maintaining profitability as usage explodes. 3. Agentic AI Models shifted CPU:GPU Ratio to 1:1 toward 3-5:1 with Explosively Token-Hungry Workloads Agentic AI (autonomous, multi-step agents with planning, tool use, iteration, and self-correction) is fundamentally more compute and token intensive than conversational or single-turn generative AI. Agentic AI. autonomous, multi-step workflows with orchestration, tool use, parallel agents, data movement, and enterprise integration has dramatically increased the importance of strong host CPUs alongside GPUs. This shifts the CPU-to-GPU ratio higher and makes balanced systems critical toward 1:1 to 5:1 as enterprises testing more than 5-10 agents. AMD EPYC Venice excels ~Leadership core density (up to 256 Zen 6 cores per socket) for running many agents in parallel, orchestration layers, and high-throughput control-plane tasks. ~Superior performance-per-core and power efficiency ( up to 2.1x higher perf/core and 2.26x better SPECpower vs. NVIDIA Grace in benchmarks). ~Tight integration in Helios: One Venice CPU + multiple MI450 GPUs per node, enabling efficient data feeding to GPUs ("zero-copy"), parallel execution, and full rack utilization for complex agentic loops. Hyperscalers (Meta, Microsoft, Amazon, Google, Softbank) and AI natives (OpenAI, Anthropic...) are adopting high-core EPYC at scale specifically for these agentic demands, as CPUs now handle a larger share of non-model work (orchestration, policy enforcement, tool calls). This complements AMD’s lower-cost GPUs for overall TCO wins. ~Agents often generate 10–100x+ more tokens per task due to iterative reasoning chains, multiple tool calls, verification loops, and long-context orchestration. ~Goldman Sachs forecasts token consumption multiplying 24x by 2030 (to 120 quadrillion tokens/month) largely driven by agentic adoption in consumer and enterprise. ~Enterprise data shows agent-pattern workloads growing at 680% annualized rates, projected to surpass conversational AI in token volume by Q3 2026. ~Daily enterprise agent token consumption is already in the billions, with complex workflows (coding, workflows, analysis) amplifying this dramatically. 4. Competitive Edge: Winning Customers from Anthropic Anthropic’s Claude models (especially Opus/Sonnet) excel in complex reasoning and agentic coding, commanding premium positioning. However, their higher underlying costs (heavier reliance on third-party cloud with margins) limit pricing flexibility compared to OpenAI’s owned Helios capacity. Anthropic is on track to generate $10.9 billion in Q2 revenue. The company expects to achieve its first-ever quarterly adjusted operating profit of $559 million. However, sustaining full-year profitability remains challenging due to immense computing and model training costs The truth is, Anthropic has no choice but to buy as much $AMD chips as possible if they want to compete with OpenAI or get investors attention. This 5% adjusted operating profit to revenue ratio is just pathetic. Current pricing dynamics (2026): OpenAI already undercuts on many tiers ( flagship output tokens significantly cheaper than equivalent Claude Opus). Nano/mini models offer 5–10x advantages for volume work. Anthropic holds edges in long-context flat pricing and certain reasoning quality. OpenAI after Helios Rack Ownership, At $0.0003–$0.0005/M effective costs, OpenAI gains massive headroom to: ~Aggressively discount high-volume agentic tiers or bundles. ~Offer “unlimited” enterprise plans or usage-based models that Anthropic struggles to match without margin erosion. ~Target cost-sensitive, high-throughput agent deployments (dev tools, automation platforms) where token bills explode. Enterprises facing $ millions in monthly agentic bills will migrate to the provider delivering better economics at scale. OpenAI’s combination of strong models (o-series reasoning) + lowest TCO positions it to erode Anthropic’s enterprise share, especially as agentic becomes the dominant token consumer. Cheaper tokens expand the total addressable market dramatically. This feeds the data/model improvement loop, justifying further capex. AMD benefits from proven scale pulling in more customers (Meta, Oracle, Microsfot, Amazon, Softbank, TensorWave, LumaAI ... already aligned on Helios). Conclusion: Dr. Lisa Su has been laser focused on inference economics since at least 2022–2023, repeatedly emphasizing that the real battleground for AI scalability would be TCO, power efficiency (TDP), and ultimately tokens per dollar and per watt not just raw training FLOPS. While many viewed inference as a secondary, commoditized workload, Dr. Su architected AMD’s roadmap around rack-scale systems optimized for high-volume, sustained inference that would dominate as models matured and usage exploded. Helios represents the culmination of that multi-year bet: a fully integrated, open platform designed precisely for the economics of massive token throughput. This deep, strategic partnership with OpenAI starting with the 1GW Helios deployment in H2 2026 and scaling to 6GW, is the embodiment of that shared vision. Both companies foresaw a future where agentic AI models evolve to become extraordinarily token-hungry: autonomous agents executing complex, iterative workflows with planning, tool use, verification loops, and long-context reasoning. These workloads can consume 100x+ more tokens per task than traditional chat or single-turn generation, driving exponential demand as capabilities improve and enterprises deploy them at scale. By owning and optimizing this massive Helios capacity at GW scale, OpenAI achieves inference costs as low as $0.0003–$0.0005 per million tokens. This structural cost advantage allows OpenAI to absorb the coming token explosion profitably, dramatically lower effective pricing for enterprises, and win high-volume agentic workloads from higher-cost competitors like Anthropic. What was once a prohibitive monthly token bill becomes an affordable accelerator for productivity and innovation. The OpenAI-AMD alliance validates Dr. Su’s prescient strategy and turns the Agentic flywheel into reality: Collapsing inference costs → explosive token consumption → richer data and better models → accelerate greater demand. This partnership doesn’t just address today’s economics, it positions both leaders at the center of the infrastructure buildout that will power AI’s next decade. By delivering the lowest inference economics at scale, OpenAI not only solves enterprise bill pain but gains a decisive weapon to win share from higher-cost rivals like Anthropic. And that is why OpenAI and $META will deploy EPYC Dense Rack Not Financial Advice! DYOR! Research Purpose Only!

$AMD| The FOMO to buy AMD Chips is NOW 🧵 Not Financial Advice! DYOR! Research Purpose Only! The Inference Queen is the biggest winner in Agentic AI where all other CPUs are struggling to compete with a 2yr old EPYC Turin and EPYC Venice is in mass production phase. AMD stresses deployability today on standard x86 platforms (no proprietary architectures required), full software compatibility, and open standards. This positions Venice + Helios as a practical, high-density alternative to competing solutions while underscoring that agentic AI shifts the balance toward CPU-rich racks alongside GPUs, and most importantly, lowering the cost of token to accelerate adoption and innovation. Context: The Wall Street Journal yesterday came out with an article that OpenAI is condiering drasstically lowering the token prices to win more customers from Anthropic. The narrative "they" are trying to exacerbate the current AI selloff won't last long. This is a fundamental misunderstanding of what is going on, or what I already discussed for months and years. Followers and Subscribers already knew this for years, that this day would come, where token cost will bcome the central discussion among enterprises as there is no such thing as unlimited budget or Tokenmaxxing when they use $NVDA chips or In-house Hyperscalers chips. I will link various threads if you are interested in understanding the full picture from supply chain to recent TSMC Rapid 2nm expansion up to 12 Fabs total by 2027/2028. Hyperscalers and AI natives effectively have no choice but to buy more AMD system for Agentic AI as leadership in economical, power-aware, high-volume internal + agentic use. However, due to supply constraints where Supply is far behind Demand, this makes multi-vendor reality along with in-house chips drive faster industry progress, lower overall costs, and better sustainability. NVIDIA’s Vera Rubin cannot compete with a 2 years old EPYC Turin, but AMD under Dr. Lisa Su has engineered the lowest cost-per-million-tokens, highly competitive energy-efficient solutions, and superior CPU orchestration for agentic AI at scale with Helios. Dr. Su has championed this shift since at least 2023, foreseeing the rise of agentic workflows that demand far more orchestration, parallel agents, and balanced compute well before the industry fully embraced it. Her long-term vision of AI moving from simple prompts to always on, multi-agent systems has driven AMD’s investments in high-core EPYC CPUs and integrated rack-scale solutions, perfectly positioning the company for today’s realities. The OpenAI-AMD 1GW Helios deployment (starting H2 2026) represents a pivotal vertical integration move that directly supercharges the inference economics. This isn't incremental; it's a structural shift toward ownership of massive, optimized rack-scale capacity, enabling the lowest token costs and triggering the enterprise adoption flywheel. We need to be honest, $AMD is the only company that made a big bet on Inference since the day Chatgpt became sensational where $NVDA and others were betting big on Training. At the end of the day, Token bill from Anthropic has to obey economics. Meaning the bills rise, companies have to get more out of it to justify the cost. It cannot be an unlimited inference budget, and it has to show up on efficiency, profitability and operating leverage. 1. Tokenomics After you understand this, you will understand why Citi cited Anthropic is likely to sign a deal with $AMD along with Hyperscalers, AI Labs, Sovereign AI like Softbank 5GW in France and many other countries. However, OpenAI and $META are now wanting faster deployment, and they are AMD shareholders now, they have prioritized allocation. Anthropic and Hyperscalers just cannot compete when Helios Rack lower token cost to$0.0003–$0.0005 per million tokens at GW scale. Cost to build 1GW data center 1GW Helios Rack full build is estimated $30-$35B 1GW Rubin Rack full build is estimated $45-$55B Inference (Cost per Million Tokens) ~$NVDA B200 / HGX: ~$0.02–$0.08 on optimized workloads (FP4/MXFP4, speculative decoding). Significant improvement over Hopper but still premium-priced. GB200 NVL72 rack-scale: $0.05–$0.25+ ~$AMD Helios Racks: $0.0003-$0.0005 per M tokens, dramatically lower than NVIDIA equivalents in owned infra. MI355X node-level: Up to 40% more tokens per dollar vs. competing solutions ( B200), driven by higher memory capacity (up to 288GB+ HBM), strong bandwidth, and lower acquisition costs. Training ~$NVDA Rubin Rack is estimated $0.7-$1.2/M Tokens ~$AMD Helios Rack is estimated $0.65-$1.0/M Tokens Now, OpenAI, META and Hyperscalers can lower Inference cost even further with $AMD EPYC Venice "dense rack" or Agentic AI Rack. AMD published a detailed technical blog emphasizing that the future of agentic AI autonomous, multi-step AI systems requiring heavy orchestration, databases, caching, APIs, and control planes demands massive CPU-dense rack-scale infrastructure, not just GPUs. The catalyst prominently positions their upcoming 6th Gen EPYC "Venice" processors as the key enabler for next-generation dense racks, delivering leadership throughput under real-world power, cooling, and density constraints. ~EPYC Venice (Zen 6 architecture, up to 256 cores / 512 threads per socket) is projected to deliver exceptional rack-level performance. In AMD’s modeled 100 kW rack comparisons, Venice-powered systems are expected to achieve ~3.30x the throughput of NVIDIA’s Vera (88-core Olympus) baseline across a broad mix of agentic-supporting workloads. ~This builds on current-generation 5th Gen EPYC "Turin" (up to 192 cores), which already delivers ~2.37x rack throughput vs. Vera and ~1.6x vs. Intel’s Xeon 6980P (128 cores). ~ Liquid-cooled Turin deployments already support >27,000 CPU cores per rack today. Venice is architected to push this beyond 36,000 cores in the same rack class, dramatically increasing concurrent agent capacity and overall infrastructure efficiency. 2. Ownership vs renting compute from Hyperscalers matter to OpenAI and only owning $AMD chips can meaningfully lower token cost for enterprises. ~Eliminates cloud overhead: No provider margins, utilization buffers, or egress fees. Direct control over power contracts, cooling, scheduling, and orchestration at dedicated facilities. ~Helios optimizations at GW scale: Rack-level density (1.4+ exaFLOPS FP8 per rack), high HBM4 bandwidth, EPYC orchestration for agentic workloads, and superior TCO/TDP. AMD's long-standing focus on tokens per dollar/watt shines here 20-40%+ efficiency edges in inference-heavy scenarios. ~At 1GW+ optimized deployment, inference hits $0.0003–$0.0005 per million tokens (community/analyst models tied to Helios metrics). This is dramatically lower than typical rented/cloud equivalents, especially for high-volume output tokens in agentic flows. High token bills today, enterprises running heavy agentic/coding/analysis workloads can face $50-100M+/month at current API rates (flagship models $5-30+/M output, scaled to massive volumes). Post-Helios compression, same volume will drop to $10-15M/month (or better) via lower underlying costs passed through as pricing flexibility, volume tiers, caching, or batch discounts. ROI thresholds collapse. More companies greenlight pilots → production → massive scaling. Agentic AI (autonomous workflows) multiplies token demand exponentially, but affordability removes the friction. OpenAI gains flexibility, Unlike more cloud-dependent rivals (Anthropic), they can lower effective pricing, offer aggressive enterprise bundles, or absorb volume without margin destruction directly tackling "high token bill" complaints while maintaining profitability as usage explodes. 3. Agentic AI Models shifted CPU:GPU Ratio to 1:1 toward 3-5:1 with Explosively Token-Hungry Workloads Agentic AI (autonomous, multi-step agents with planning, tool use, iteration, and self-correction) is fundamentally more compute and token intensive than conversational or single-turn generative AI. Agentic AI. autonomous, multi-step workflows with orchestration, tool use, parallel agents, data movement, and enterprise integration has dramatically increased the importance of strong host CPUs alongside GPUs. This shifts the CPU-to-GPU ratio higher and makes balanced systems critical toward 1:1 to 5:1 as enterprises testing more than 5-10 agents. AMD EPYC Venice excels ~Leadership core density (up to 256 Zen 6 cores per socket) for running many agents in parallel, orchestration layers, and high-throughput control-plane tasks. ~Superior performance-per-core and power efficiency ( up to 2.1x higher perf/core and 2.26x better SPECpower vs. NVIDIA Grace in benchmarks). ~Tight integration in Helios: One Venice CPU + multiple MI450 GPUs per node, enabling efficient data feeding to GPUs ("zero-copy"), parallel execution, and full rack utilization for complex agentic loops. Hyperscalers (Meta, Microsoft, Amazon, Google, Softbank) and AI natives (OpenAI, Anthropic...) are adopting high-core EPYC at scale specifically for these agentic demands, as CPUs now handle a larger share of non-model work (orchestration, policy enforcement, tool calls). This complements AMD’s lower-cost GPUs for overall TCO wins. ~Agents often generate 10–100x+ more tokens per task due to iterative reasoning chains, multiple tool calls, verification loops, and long-context orchestration. ~Goldman Sachs forecasts token consumption multiplying 24x by 2030 (to 120 quadrillion tokens/month) largely driven by agentic adoption in consumer and enterprise. ~Enterprise data shows agent-pattern workloads growing at 680% annualized rates, projected to surpass conversational AI in token volume by Q3 2026. ~Daily enterprise agent token consumption is already in the billions, with complex workflows (coding, workflows, analysis) amplifying this dramatically. 4. Competitive Edge: Winning Customers from Anthropic Anthropic’s Claude models (especially Opus/Sonnet) excel in complex reasoning and agentic coding, commanding premium positioning. However, their higher underlying costs (heavier reliance on third-party cloud with margins) limit pricing flexibility compared to OpenAI’s owned Helios capacity. Anthropic is on track to generate $10.9 billion in Q2 revenue. The company expects to achieve its first-ever quarterly adjusted operating profit of $559 million. However, sustaining full-year profitability remains challenging due to immense computing and model training costs The truth is, Anthropic has no choice but to buy as much $AMD chips as possible if they want to compete with OpenAI or get investors attention. This 5% adjusted operating profit to revenue ratio is just pathetic. Current pricing dynamics (2026): OpenAI already undercuts on many tiers ( flagship output tokens significantly cheaper than equivalent Claude Opus). Nano/mini models offer 5–10x advantages for volume work. Anthropic holds edges in long-context flat pricing and certain reasoning quality. OpenAI after Helios Rack Ownership, At $0.0003–$0.0005/M effective costs, OpenAI gains massive headroom to: ~Aggressively discount high-volume agentic tiers or bundles. ~Offer “unlimited” enterprise plans or usage-based models that Anthropic struggles to match without margin erosion. ~Target cost-sensitive, high-throughput agent deployments (dev tools, automation platforms) where token bills explode. Enterprises facing $ millions in monthly agentic bills will migrate to the provider delivering better economics at scale. OpenAI’s combination of strong models (o-series reasoning) + lowest TCO positions it to erode Anthropic’s enterprise share, especially as agentic becomes the dominant token consumer. Cheaper tokens expand the total addressable market dramatically. This feeds the data/model improvement loop, justifying further capex. AMD benefits from proven scale pulling in more customers (Meta, Oracle, Microsfot, Amazon, Softbank, TensorWave, LumaAI ... already aligned on Helios). Conclusion: Dr. Lisa Su has been laser focused on inference economics since at least 2022–2023, repeatedly emphasizing that the real battleground for AI scalability would be TCO, power efficiency (TDP), and ultimately tokens per dollar and per watt not just raw training FLOPS. While many viewed inference as a secondary, commoditized workload, Dr. Su architected AMD’s roadmap around rack-scale systems optimized for high-volume, sustained inference that would dominate as models matured and usage exploded. Helios represents the culmination of that multi-year bet: a fully integrated, open platform designed precisely for the economics of massive token throughput. This deep, strategic partnership with OpenAI starting with the 1GW Helios deployment in H2 2026 and scaling to 6GW, is the embodiment of that shared vision. Both companies foresaw a future where agentic AI models evolve to become extraordinarily token-hungry: autonomous agents executing complex, iterative workflows with planning, tool use, verification loops, and long-context reasoning. These workloads can consume 100x+ more tokens per task than traditional chat or single-turn generation, driving exponential demand as capabilities improve and enterprises deploy them at scale. By owning and optimizing this massive Helios capacity at GW scale, OpenAI achieves inference costs as low as $0.0003–$0.0005 per million tokens. This structural cost advantage allows OpenAI to absorb the coming token explosion profitably, dramatically lower effective pricing for enterprises, and win high-volume agentic workloads from higher-cost competitors like Anthropic. What was once a prohibitive monthly token bill becomes an affordable accelerator for productivity and innovation. The OpenAI-AMD alliance validates Dr. Su’s prescient strategy and turns the Agentic flywheel into reality: Collapsing inference costs → explosive token consumption → richer data and better models → accelerate greater demand. This partnership doesn’t just address today’s economics, it positions both leaders at the center of the infrastructure buildout that will power AI’s next decade. By delivering the lowest inference economics at scale, OpenAI not only solves enterprise bill pain but gains a decisive weapon to win share from higher-cost rivals like Anthropic. And that is why OpenAI and $META will deploy EPYC Dense Rack Not Financial Advice! DYOR! Research Purpose Only!

Mike

84,951 görüntüleme • 3 ay önce

This battery is about to change the world in 3 months, or make this guy a fool | Fred Lambert, Hacker News Donut Lab lit the EV and energy storage industry on fire last week with its announcement of a 400 Wh/kg solid-state battery cell that can last for 100 years. At face value, if true, we are looking at the single most disruptive announcement in the history of the electric vehicle industry and energy storage as a whole. We aren’t just talking about a better motorcycle battery. If the claims of a 5-minute charge, 100,000-cycle life, and ~400 Wh/kg energy density are accurate and scalable, as Donut Lab claims, this is the holy grail of energy storage. Battery breakthrough announcements generally don’t catch fire like this, but Donut Lab’s did because it said that the cell was already in production and will be in a production vehicle, Verge’s electric motorcycle, this quarter. It gave credibility to the claim, pushing everyone to report on it. Now, we have interviewed Donut Lab’s CEO and investigated the technology. At this point, it looks like either this battery changes the world within the next 3 months, or it will make the CEO look like a fool. In this article, we discuss the impact of the battery, whether real or not, as well as clues about the secret sauce behind its chemistry. The Holy Grail of Energy Storage Consider the implications. A battery that lasts 100,000 cycles is effectively immortal in human terms. You could charge it every single day for 270 years, and it would still be working. It means the battery outlives the vehicle, not just once, but ten times over. It changes the economics of transportation entirely: you buy the battery once, and you swap it into your next five cars. The power density required for a 5-minute charge and the 400 Wh/kg of energy density opens the door to commercial electric aviation, a sector currently strangled by the weight and slow charging speeds of lithium-ion. It solves the grid storage problem by offering a medium that doesn’t degrade, meaning utility companies could amortize the cost over a century rather than a decade. If this is real, the internal combustion engine didn’t just die today; it was buried 100 feet deep, and every other battery is not far behind. But, and this is a massive “but”, extraordinary claims require extraordinary proof, and Donut Lab has yet to release that proof. And that brings us to the man making them. The Man Betting His Reputation I spoke with Marko Lehtimäki, the CEO of Donut Lab and Chairman of Verge Motorcycles. My goal was simple: ask him about the chemistry behind his battery and, if that doesn’t work, look him in the eye and figure out if he’s selling vaporware or if he’s sitting on the breakthrough of the century. Marko isn’t a random guy shouting about a battery breakthrough that will change the world. He is a legit entrepreneur. A computer scientist who built a no-coding app builder years before “vibe coding” was even a thing and sold it to SAP. After the successful exit, he became an investor and serial entrepreneur with his biggest, or most well-known, company being Verge Motorcycles, which has real products on the road. By announcing that this “miracle battery” is already in production and will be shipping in customer vehicles within 10 weeks, he is betting his entire personal reputation on this technology. If he misses this timeline or if the specs are fake, Donut Labs and Verge Motorcycles might not survive the credibility loss. He has a lot to lose here. In my article about the battery announcement last week, I noted that Marko’s presentation was incredible. He basically described a perfect battery: record energy density, incredible charge rate, unprecedented longevity, no rare metals, a cost lower than traditional Li-ion cells, and in scalable production right now. Sounds too good to be true? The only thing he didn’t share was details about the chemistry, beyond saying it doesn’t use lithium or other rare metals. What’s the point of protecting the chemistry if the battery is already in production and it will be in a product shipped this quarter? If that’s true, the battery will be reverse-engineered before the snow completely melts. We discussed it with Marko during our interview. His logic is that once the bikes ship, competitors will tear them down and figure it out anyway. But that won’t happen for another 10 weeks or so, and the head start is critical for a technology this disruptive. In the meantime, Donut Lab’s goal with the announcement was to get the attention of OEMs and ship them battery packs for validation. Marko said: We are right now shipping demo packs to OEMs under NDAs and under tight disclosures so that they can test that all of that is true, which serves our business very well [better than disclosing the chemistry]. But these programs with OEMs are likely to take a long time before they become public. Shorter term, there’s Verge Motorcycles shipping bikes with the battery by the end of the quarter. Before that, Marko also said that we should soon see third-party testing of those cells: We rather right now ship it to authorized research and science center that tests everything without opening it and telling everybody what’s in there. In short, we should have a good idea whether the claims are true or not in just a few weeks no matter what. What does Marko, or Donut Lab, have to gain by lying about this? I also discussed this with Marko and the only thing I could come up with is if he happens to be raising capital right now, but he shut that down: There are a million investors chasing us right now, but we are literally not talking to anybody. We tell investors that we can discuss terms after we have done all our disclosures. Marko insisted that Donut Lab is not taking any investment until they have proven their cells work. In short, it’s hard to find an upside for Donut Lab in making this announcement if the claims are not true. It doesn’t mean that they are, but it makes you think. The Investigation: What Is the “Donut Battery”? So, what is the secret sauce? Marko wouldn’t say, but after digging into public records, supply chains, and research papers, I believe we have a pretty good idea. Let me preface this by saying that I’m not a chemist or physicist, but I’ve been a journalist covering electric vehicles for more than a decade, and I’m pretty good at connecting the dots, and in this case, I’ve had the help of a couple of great sources, too. I’m not saying that this is the Donut Lab battery, but since they are not sharing much, we have to speculate, and all evidence points to a Finnish nanotechnology startup called Nordic Nano and its Chief Scientist, Dr. Bela Bhuskute. Donut Lab invested in Nordic Nano in October 2025, just months before this announcement. At the time of writing this, the press release has fewer than 200 views. The announcement went under the radar, and while Marko said that Nordic Nano is more of a “solar company” during our interview, the announcement mentions both solar and energy storage. Dr. Bhuskute’s research at Tampere University focuses on amorphous Titanium Dioxide nanostructures, which could benefit many different technologies, including batteries. It fits the “miracle” specs perfectly: - 100,000 Cycles: Traditional solid-state batteries are crystalline (like a brick wall) and crack when ions rush in. Dr. Bhuskute’s amorphous Titanium Dioxide is disordered (like a sponge) and “breathes,” allowing it to expand and contract without breaking. - 5-Minute Charge: This chemistry stores energy via “pseudocapacitance,” which is basically like Velcro. Ions stick to the surface almost instantly rather than having to burrow deep inside the material. - The Manufacturing: Nordic Nano uses a “nanofluid” printing process for its solar product using the technology. This aligns with Donut Lab’s description of a “clay-like” material that enables an easier manufacturing process. Some call this “battery printing”, which could explain Donut Lab’s ability to bring this to production in record time. When I asked Marko for the volumetric energy density (Wh/L), he claimed he “couldn’t remember”. Volumetric energy density is one of the few specs that Donut Lab hasn’t released. This battery is lighter than lithium-ion, but it could be bigger due to the amorphous nature of the titanium dioxide. However, the CEO claimed it has a higher volumetric density than traditional Li-ion batteries, without providing a specific number. If that’s true, not only could electric vehicles and energy storage switch to this new chemistry, but even personal electronics, such as smartphones. In 2025, Nordic Nano has been making moves, including securing a former large retail location in Imatra, Finland, near the Russian border: It could be where the company has set up production. Following investment from the Finnish government, Nordic Nano had to elaborate a bit on its products and confirmed that it is working on “solar energy systems and energy storage solutions”: The company’s range of products includes two product families: solar energy systems and energy storage solutions: The ultra-thin and flexible solar film collects twice the amount of energy compared to traditional silicon-based solar panels. Solid-state salt batteries are manufactured by printing from nanofluid, which enables the efficient use of space and the production of batteries in varying shapes. Furthermore, the company confirmed that it is using a “screenprinting” manufacturing method. This is not new. Other companies have produced battery cells with this technology with varying degrees of success. It appears that the bet is that the amorphous rather than crystalized titanium dioxide nanostructure could be more easily adapted and scaled with this manufacturing technology. Electrek’s Take I’m naturally skeptical, and this screams “too good to be true”, but I can’t find anything that categorically rejects the claims. I get battery breakthrough announcements in my inbox every week, and most of the time they never amount to anything. If I decide to spend some time researching them and talking to experts, I generally quickly hit a problem or two that make them commercially unviable. This announcement is different. We can’t really investigate the actual breakthrough; we can only speculate about it, since it is guarded. Marko’s logic for guarding the chemistry is sound, and the incentives to lie about what they have aren’t clear if he is not currently raising money. Then, because they claim this is already in production and will be in a deliverable product within weeks, we will know whether the claims are true in short order, and their reputations, especially Marko’s, are on the line. During my interview, Marko didn’t seem too worried about it. It doesn’t sound like someone who needs to quickly figure out how to deliver this, but rather someone who has a couple of aces in their hand and is looking to maximize them. It’s also strange that this innovation and then production quickly comes from a relatively small company. I thought researching Donut Lab would make me more skeptical about the claims, but it’s the contrary. It confirms that their technology stems from years of research, backed by university and government funding for its commercialization. Could it be that this critical research went under the radar and a small electric motorcycle startup in need of a significant bump in energy density stumbled upon it? Then, a savvy entrepreneur quickly found a way to optimize the impact of this potentially groundbreaking tech by spinning out a startup from the motorcycle company to market the battery to a broader market. Maybe? This could be real, or it could be hype. Again, I’m still skeptical, but I can’t point to anything specific that would disprove any claim made about this miracle battery. Again, if this is true, we are talking about a complete reset of the entire energy and transportation sectors. Donut Lab would become one of the biggest companies in the world. A Nobel Prize would be coming to Dr. Bhuskute and her colleagues in the near future. If it’s not, Marko and Donut Lab’s reputation would be destroyed. There might also be a middle conclusion where the battery is nearly as good as they claim, but when you ramp up production, other problems arise, such as scrap, which has been the undoing of another company that recently tried screenprinting batteries. Who knows? But it sounds like we should find out soon. Within weeks, we should get independent verifications of the specs. Then the bikes get delivered within months. You can fake a presentation, but there are things you can’t fake.

This battery is about to change the world in 3 months, or make this guy a fool | Fred Lambert, Hacker News Donut Lab lit the EV and energy storage industry on fire last week with its announcement of a 400 Wh/kg solid-state battery cell that can last for 100 years. At face value, if true, we are looking at the single most disruptive announcement in the history of the electric vehicle industry and energy storage as a whole. We aren’t just talking about a better motorcycle battery. If the claims of a 5-minute charge, 100,000-cycle life, and ~400 Wh/kg energy density are accurate and scalable, as Donut Lab claims, this is the holy grail of energy storage. Battery breakthrough announcements generally don’t catch fire like this, but Donut Lab’s did because it said that the cell was already in production and will be in a production vehicle, Verge’s electric motorcycle, this quarter. It gave credibility to the claim, pushing everyone to report on it. Now, we have interviewed Donut Lab’s CEO and investigated the technology. At this point, it looks like either this battery changes the world within the next 3 months, or it will make the CEO look like a fool. In this article, we discuss the impact of the battery, whether real or not, as well as clues about the secret sauce behind its chemistry. The Holy Grail of Energy Storage Consider the implications. A battery that lasts 100,000 cycles is effectively immortal in human terms. You could charge it every single day for 270 years, and it would still be working. It means the battery outlives the vehicle, not just once, but ten times over. It changes the economics of transportation entirely: you buy the battery once, and you swap it into your next five cars. The power density required for a 5-minute charge and the 400 Wh/kg of energy density opens the door to commercial electric aviation, a sector currently strangled by the weight and slow charging speeds of lithium-ion. It solves the grid storage problem by offering a medium that doesn’t degrade, meaning utility companies could amortize the cost over a century rather than a decade. If this is real, the internal combustion engine didn’t just die today; it was buried 100 feet deep, and every other battery is not far behind. But, and this is a massive “but”, extraordinary claims require extraordinary proof, and Donut Lab has yet to release that proof. And that brings us to the man making them. The Man Betting His Reputation I spoke with Marko Lehtimäki, the CEO of Donut Lab and Chairman of Verge Motorcycles. My goal was simple: ask him about the chemistry behind his battery and, if that doesn’t work, look him in the eye and figure out if he’s selling vaporware or if he’s sitting on the breakthrough of the century. Marko isn’t a random guy shouting about a battery breakthrough that will change the world. He is a legit entrepreneur. A computer scientist who built a no-coding app builder years before “vibe coding” was even a thing and sold it to SAP. After the successful exit, he became an investor and serial entrepreneur with his biggest, or most well-known, company being Verge Motorcycles, which has real products on the road. By announcing that this “miracle battery” is already in production and will be shipping in customer vehicles within 10 weeks, he is betting his entire personal reputation on this technology. If he misses this timeline or if the specs are fake, Donut Labs and Verge Motorcycles might not survive the credibility loss. He has a lot to lose here. In my article about the battery announcement last week, I noted that Marko’s presentation was incredible. He basically described a perfect battery: record energy density, incredible charge rate, unprecedented longevity, no rare metals, a cost lower than traditional Li-ion cells, and in scalable production right now. Sounds too good to be true? The only thing he didn’t share was details about the chemistry, beyond saying it doesn’t use lithium or other rare metals. What’s the point of protecting the chemistry if the battery is already in production and it will be in a product shipped this quarter? If that’s true, the battery will be reverse-engineered before the snow completely melts. We discussed it with Marko during our interview. His logic is that once the bikes ship, competitors will tear them down and figure it out anyway. But that won’t happen for another 10 weeks or so, and the head start is critical for a technology this disruptive. In the meantime, Donut Lab’s goal with the announcement was to get the attention of OEMs and ship them battery packs for validation. Marko said: We are right now shipping demo packs to OEMs under NDAs and under tight disclosures so that they can test that all of that is true, which serves our business very well [better than disclosing the chemistry]. But these programs with OEMs are likely to take a long time before they become public. Shorter term, there’s Verge Motorcycles shipping bikes with the battery by the end of the quarter. Before that, Marko also said that we should soon see third-party testing of those cells: We rather right now ship it to authorized research and science center that tests everything without opening it and telling everybody what’s in there. In short, we should have a good idea whether the claims are true or not in just a few weeks no matter what. What does Marko, or Donut Lab, have to gain by lying about this? I also discussed this with Marko and the only thing I could come up with is if he happens to be raising capital right now, but he shut that down: There are a million investors chasing us right now, but we are literally not talking to anybody. We tell investors that we can discuss terms after we have done all our disclosures. Marko insisted that Donut Lab is not taking any investment until they have proven their cells work. In short, it’s hard to find an upside for Donut Lab in making this announcement if the claims are not true. It doesn’t mean that they are, but it makes you think. The Investigation: What Is the “Donut Battery”? So, what is the secret sauce? Marko wouldn’t say, but after digging into public records, supply chains, and research papers, I believe we have a pretty good idea. Let me preface this by saying that I’m not a chemist or physicist, but I’ve been a journalist covering electric vehicles for more than a decade, and I’m pretty good at connecting the dots, and in this case, I’ve had the help of a couple of great sources, too. I’m not saying that this is the Donut Lab battery, but since they are not sharing much, we have to speculate, and all evidence points to a Finnish nanotechnology startup called Nordic Nano and its Chief Scientist, Dr. Bela Bhuskute. Donut Lab invested in Nordic Nano in October 2025, just months before this announcement. At the time of writing this, the press release has fewer than 200 views. The announcement went under the radar, and while Marko said that Nordic Nano is more of a “solar company” during our interview, the announcement mentions both solar and energy storage. Dr. Bhuskute’s research at Tampere University focuses on amorphous Titanium Dioxide nanostructures, which could benefit many different technologies, including batteries. It fits the “miracle” specs perfectly: - 100,000 Cycles: Traditional solid-state batteries are crystalline (like a brick wall) and crack when ions rush in. Dr. Bhuskute’s amorphous Titanium Dioxide is disordered (like a sponge) and “breathes,” allowing it to expand and contract without breaking. - 5-Minute Charge: This chemistry stores energy via “pseudocapacitance,” which is basically like Velcro. Ions stick to the surface almost instantly rather than having to burrow deep inside the material. - The Manufacturing: Nordic Nano uses a “nanofluid” printing process for its solar product using the technology. This aligns with Donut Lab’s description of a “clay-like” material that enables an easier manufacturing process. Some call this “battery printing”, which could explain Donut Lab’s ability to bring this to production in record time. When I asked Marko for the volumetric energy density (Wh/L), he claimed he “couldn’t remember”. Volumetric energy density is one of the few specs that Donut Lab hasn’t released. This battery is lighter than lithium-ion, but it could be bigger due to the amorphous nature of the titanium dioxide. However, the CEO claimed it has a higher volumetric density than traditional Li-ion batteries, without providing a specific number. If that’s true, not only could electric vehicles and energy storage switch to this new chemistry, but even personal electronics, such as smartphones. In 2025, Nordic Nano has been making moves, including securing a former large retail location in Imatra, Finland, near the Russian border: It could be where the company has set up production. Following investment from the Finnish government, Nordic Nano had to elaborate a bit on its products and confirmed that it is working on “solar energy systems and energy storage solutions”: The company’s range of products includes two product families: solar energy systems and energy storage solutions: The ultra-thin and flexible solar film collects twice the amount of energy compared to traditional silicon-based solar panels. Solid-state salt batteries are manufactured by printing from nanofluid, which enables the efficient use of space and the production of batteries in varying shapes. Furthermore, the company confirmed that it is using a “screenprinting” manufacturing method. This is not new. Other companies have produced battery cells with this technology with varying degrees of success. It appears that the bet is that the amorphous rather than crystalized titanium dioxide nanostructure could be more easily adapted and scaled with this manufacturing technology. Electrek’s Take I’m naturally skeptical, and this screams “too good to be true”, but I can’t find anything that categorically rejects the claims. I get battery breakthrough announcements in my inbox every week, and most of the time they never amount to anything. If I decide to spend some time researching them and talking to experts, I generally quickly hit a problem or two that make them commercially unviable. This announcement is different. We can’t really investigate the actual breakthrough; we can only speculate about it, since it is guarded. Marko’s logic for guarding the chemistry is sound, and the incentives to lie about what they have aren’t clear if he is not currently raising money. Then, because they claim this is already in production and will be in a deliverable product within weeks, we will know whether the claims are true in short order, and their reputations, especially Marko’s, are on the line. During my interview, Marko didn’t seem too worried about it. It doesn’t sound like someone who needs to quickly figure out how to deliver this, but rather someone who has a couple of aces in their hand and is looking to maximize them. It’s also strange that this innovation and then production quickly comes from a relatively small company. I thought researching Donut Lab would make me more skeptical about the claims, but it’s the contrary. It confirms that their technology stems from years of research, backed by university and government funding for its commercialization. Could it be that this critical research went under the radar and a small electric motorcycle startup in need of a significant bump in energy density stumbled upon it? Then, a savvy entrepreneur quickly found a way to optimize the impact of this potentially groundbreaking tech by spinning out a startup from the motorcycle company to market the battery to a broader market. Maybe? This could be real, or it could be hype. Again, I’m still skeptical, but I can’t point to anything specific that would disprove any claim made about this miracle battery. Again, if this is true, we are talking about a complete reset of the entire energy and transportation sectors. Donut Lab would become one of the biggest companies in the world. A Nobel Prize would be coming to Dr. Bhuskute and her colleagues in the near future. If it’s not, Marko and Donut Lab’s reputation would be destroyed. There might also be a middle conclusion where the battery is nearly as good as they claim, but when you ramp up production, other problems arise, such as scrap, which has been the undoing of another company that recently tried screenprinting batteries. Who knows? But it sounds like we should find out soon. Within weeks, we should get independent verifications of the specs. Then the bikes get delivered within months. You can fake a presentation, but there are things you can’t fake.

Owen Gregorian

123,359 görüntüleme • 8 ay önce

When Elon Musk beams in virtually for a high-stakes fireside chat with JPMorgan Chase CEO Jamie Dimon, the conversation goes completely out of this world. The discussion was packed with massive milestones—from the bombshell that SpaceX is going public to plans for lunar AI data centers and the urgent need for the Terafab chip revolution. Here is the ultimate breakdown of their discussion: 💵 SpaceX has been self-funding and cash-flow positive for a decade Before the decision to go public, SpaceX didn't actually need to raise money to survive. The company has been cash-flow positive since around 2014–2015, meaning its private equity rounds were exclusively held to provide liquidity for employees and early investors. "We've been positive cash flow for quite a long time, I think, since around 2014-2015. And we've been self-funding. In fact, in our sort of private equity rounds, they actually have not been fundraising rounds. They've been liquidity rounds for investors and employees because we give everyone at the company stock." 🚀 The upcoming capital growth phase requires massive funding The primary trigger for going public now is an unprecedented capital expenditure phase. SpaceX is preparing to deploy an immense constellation of over 100,000 Next-Gen communication satellites and construct massive AI data centers in orbit. "we are embarking on a significant capital growth phase where we're going to put in over probably 100,000 satellites, probably over 100,000 satellites, just for communications... And then we're also doing the AI data centers in space, which is another massive capital endeavor." 📡 Starlink V3 introduces a massive bandwidth breakthrough The custom chips designed by SpaceX for the V3 satellites will completely alter global communications, offering 100 times the bandwidth of the current system and slashing latency in half by operating at a lower altitude. They are so large—the size of a small bus—that Starship is the only rocket on Earth capable of launching them, carrying 50 at a time. "The version three is, depending on how you count it, 10 to 20 times more capable than the version two satellite. And there were three chips that the SpaceX chip design team taped out that are specific to this... Which means it's 100 times more bandwidth than the SpaceX's Starlink system currently on the surface. And also half the latency because the altitude will be about half altitude." 🤖 AI and robots possess an insatiable appetite for data Musk points out that expanding infrastructure into space is vital because future AI and robotic systems will demand an astronomical amount of bandwidth compared to the relatively low data transmission rates of human beings. "And the future with AI and robots is actually going to require a lot more bandwidth than we currently use. Because you can imagine like what's the bandwidth of a human? Peak bandwidth of the human is a few hundred bits per second. But bandwidth of a computer can be a trillion bits a second. So the appetite for bandwidth of AI and robots is going to be enormous." ☀️ Space solves the looming terrestrial power plant crisis Building traditional power plants on Earth faces heavy community resistance. Moving data centers into space unlocks unlimited energy generation via solar power ("star power") without disrupting Earth's environment, tapping into an energy source that accounts for 99.8% of the solar system's mass. "It's increasingly difficult to build power plants on the ground. There are very few people who want a power plant in their backyard... But actually if we go to space, we can go far beyond the electricity generation of both. In fact, this is going to sound kind of crazy. But you could actually increase human energy by a factor of a million and still be using much less than a millionth of the sun's energy." 🌕 The Moon is a 1,000-Terawatt compute launchpad While Mars remains the long-term goal, the Moon is the immediate fast-track location for massive scaling. Because it lacks an atmosphere and has low gravity, SpaceX can use electromagnetic rail guns to shoot AI data centers into deep space from the lunar surface, scaling power to an incredible 1,000 terawatts per year. "I just think that we can build a self-sustaining city on the moon faster than we could do so on Mars. And there's also the potential... you can use an electromagnetic accelerator, a rail gun or mass driver. Basically, you don't need to use rockets to do AI data centers into deep space from the moon... We can do a thousand terawatts or more from the moon." 🪐 Mars is the ultimate "fixer-upper" planet Mars is being targeted as a full-scale terraforming project. Due to its atmosphere and gravity levels, warming up the planet could eventually unlock liquid oceans and allow humans to walk around without spacesuits. "And if you warm up Mars, you could one day make Mars like Earth. And with like liquid oceans and life. And where you could walk outside without a spacesuit type of thing. So Mars is, I call Mars a fixer upper of a planet. But it's got a lot of potential." 🚂 SpaceX is the modern-day Union Pacific Railroad Musk rejects the idea that SpaceX is moving into the hospitality or hotel business for space tourism. Instead, he views the company as a foundational infrastructure provider, comparable to the historic railroads that opened up the American West. "We're kind of like Union Pacific, you know. You know, when they built Union Pacific back in the day, people thought they were crazy. Because like, why are you trying to carry all this cargo and people to California? No one's there. But now California is the biggest state in the country." ♻️ Starship's core disruption is 100% reusability The true holy grail of Starship is full reusability, which drops orbit access costs down to the mere price of fuel. Because it utilizes ultra-cheap liquid oxygen and methane, shipping cargo to space will become more economical than flying cargo across Earth's oceans on an airplane. "The fundamental breakthrough of Starship is that it will be the first orbital rocket that is fully reusable... And the propellant we use for Starship is liquid oxygen and liquid methane, which is the cheapest propellant you could possibly get... which means that you should be able to actually send cargo to space for less than the cost of cargo on an airplane going on a trans-oceanic trip." 🔄 Starship V4 targets hourly launch cadences SpaceX's engineering pipeline is aiming for staggering operational frequencies and massive payloads. While Starship V3 targets 100 tons to orbit, the upcoming V4 variant is designed to carry over 200 tons and launch on an hourly schedule. "Because Starship V3 is aiming to do 100 tons to orbit with full reusability. And then Starship V4 we're aiming for over 200 tons per mission. And then being able to launch every hour." ☁️ Orbital data centers are entirely weather-proof Space-based AI data centers are highly practical because they are simpler to construct than communication satellites. Data is beamed via lasers between satellites, and then beamed to the ground using cloud-penetrating radio frequencies that completely bypass bad weather. "The AI data center would be much simpler by comparison. Because it's really just solar power plus radiator... The connection would happen no matter what the weather is. Because once you connect via the lasers to the Starlink communication constellation, the Starlink communication to the ground uses frequencies that are cloud penetrating." 🇺🇸 The U.S. faces a catastrophic "Zero Memory Fab" crisis A major vulnerability in domestic tech infrastructure is that the U.S. currently manufactures zero high-volume computer memory chips. Even with new facilities arriving online between 2028 and 2030, domestic supply will not match the exponential requirements of AI, which is why Musk is aggressively building the Terafab. "there's not a single high volume computer memory fab in America right now. Zero. There's one being built in Idaho by Micron. But that will not reach volume production until I believe 2028. And there's something being built in New York, but they are in, I think, 29 and 30. And this is a tiny fraction of the memory that's needed... That's why we need to do the Terafab." 🧠 SpaceX will offer proprietary AI chips and software While the orbital data center network will remain an open marketplace capable of running third-party hardware like NVIDIA GPUs, Google TPUs, or Amazon Trainium, SpaceX plans to deploy its own in-house AI chips and software stack in the near future. "So if NVIDIA GPUs can be put on it, Google TPUs can be put on it, Amazon Trainium or any other chips that you want to put on, can be put on. We'll also offer our chips in the future and I think we also want to offer our software, our AI software as well in the future." 🛡️ Starshield handles critical national intelligence Musk emphasizes his deeply pro-American stance, highlighting SpaceX's specialized Starshield division as a crucial backbone for the U.S. military and national intelligence agencies. "We have a division called Starshield which provides military communications. And you know, there's some other stuff that's kind of classified, I guess. We can't be talking about that. But we are helping the Department of War and intelligence part of the government. We're a vital element of that." 👥 Executive retention fuels the mission The core leadership bench at SpaceX is defined by extreme longevity, driven by a deep collective belief in turning science fiction into reality. Top executives like Gwynne Shotwell have remained with Musk for over two decades. "I guess Gwynne was, I think, around the seventh person to join the company. And that was 2002. It's just went to like 24 years. And generally the senior executives at the company, you have a very long tenure. I think Brent Johnson's been, you see, over 15 years... because people really believe in the mission, I think they want to stay and they want to keep building it." ❤️ Character overrides IQ in leadership Reflecting on how he has evolved over 20 years, Musk notes that he has become significantly more laid back. He has also learned that a candidate's moral character and heart are just as vital to a company's success as raw intellectual horsepower. "Well, I think I'm probably more chill than I used to be... And one of the things I've found over time... is that like in terms of like recruiting people to the company and having people work with the company, like their individual abilities and their intellectual capabilities matter a lot, but it also matters if they have a good heart. It's not just about whether somebody has a certain IQ or whatever, but just are they like a good person, that matters a lot."

When Elon Musk beams in virtually for a high-stakes fireside chat with JPMorgan Chase CEO Jamie Dimon, the conversation goes completely out of this world. The discussion was packed with massive milestones—from the bombshell that SpaceX is going public to plans for lunar AI data centers and the urgent need for the Terafab chip revolution. Here is the ultimate breakdown of their discussion: 💵 SpaceX has been self-funding and cash-flow positive for a decade Before the decision to go public, SpaceX didn't actually need to raise money to survive. The company has been cash-flow positive since around 2014–2015, meaning its private equity rounds were exclusively held to provide liquidity for employees and early investors. "We've been positive cash flow for quite a long time, I think, since around 2014-2015. And we've been self-funding. In fact, in our sort of private equity rounds, they actually have not been fundraising rounds. They've been liquidity rounds for investors and employees because we give everyone at the company stock." 🚀 The upcoming capital growth phase requires massive funding The primary trigger for going public now is an unprecedented capital expenditure phase. SpaceX is preparing to deploy an immense constellation of over 100,000 Next-Gen communication satellites and construct massive AI data centers in orbit. "we are embarking on a significant capital growth phase where we're going to put in over probably 100,000 satellites, probably over 100,000 satellites, just for communications... And then we're also doing the AI data centers in space, which is another massive capital endeavor." 📡 Starlink V3 introduces a massive bandwidth breakthrough The custom chips designed by SpaceX for the V3 satellites will completely alter global communications, offering 100 times the bandwidth of the current system and slashing latency in half by operating at a lower altitude. They are so large—the size of a small bus—that Starship is the only rocket on Earth capable of launching them, carrying 50 at a time. "The version three is, depending on how you count it, 10 to 20 times more capable than the version two satellite. And there were three chips that the SpaceX chip design team taped out that are specific to this... Which means it's 100 times more bandwidth than the SpaceX's Starlink system currently on the surface. And also half the latency because the altitude will be about half altitude." 🤖 AI and robots possess an insatiable appetite for data Musk points out that expanding infrastructure into space is vital because future AI and robotic systems will demand an astronomical amount of bandwidth compared to the relatively low data transmission rates of human beings. "And the future with AI and robots is actually going to require a lot more bandwidth than we currently use. Because you can imagine like what's the bandwidth of a human? Peak bandwidth of the human is a few hundred bits per second. But bandwidth of a computer can be a trillion bits a second. So the appetite for bandwidth of AI and robots is going to be enormous." ☀️ Space solves the looming terrestrial power plant crisis Building traditional power plants on Earth faces heavy community resistance. Moving data centers into space unlocks unlimited energy generation via solar power ("star power") without disrupting Earth's environment, tapping into an energy source that accounts for 99.8% of the solar system's mass. "It's increasingly difficult to build power plants on the ground. There are very few people who want a power plant in their backyard... But actually if we go to space, we can go far beyond the electricity generation of both. In fact, this is going to sound kind of crazy. But you could actually increase human energy by a factor of a million and still be using much less than a millionth of the sun's energy." 🌕 The Moon is a 1,000-Terawatt compute launchpad While Mars remains the long-term goal, the Moon is the immediate fast-track location for massive scaling. Because it lacks an atmosphere and has low gravity, SpaceX can use electromagnetic rail guns to shoot AI data centers into deep space from the lunar surface, scaling power to an incredible 1,000 terawatts per year. "I just think that we can build a self-sustaining city on the moon faster than we could do so on Mars. And there's also the potential... you can use an electromagnetic accelerator, a rail gun or mass driver. Basically, you don't need to use rockets to do AI data centers into deep space from the moon... We can do a thousand terawatts or more from the moon." 🪐 Mars is the ultimate "fixer-upper" planet Mars is being targeted as a full-scale terraforming project. Due to its atmosphere and gravity levels, warming up the planet could eventually unlock liquid oceans and allow humans to walk around without spacesuits. "And if you warm up Mars, you could one day make Mars like Earth. And with like liquid oceans and life. And where you could walk outside without a spacesuit type of thing. So Mars is, I call Mars a fixer upper of a planet. But it's got a lot of potential." 🚂 SpaceX is the modern-day Union Pacific Railroad Musk rejects the idea that SpaceX is moving into the hospitality or hotel business for space tourism. Instead, he views the company as a foundational infrastructure provider, comparable to the historic railroads that opened up the American West. "We're kind of like Union Pacific, you know. You know, when they built Union Pacific back in the day, people thought they were crazy. Because like, why are you trying to carry all this cargo and people to California? No one's there. But now California is the biggest state in the country." ♻️ Starship's core disruption is 100% reusability The true holy grail of Starship is full reusability, which drops orbit access costs down to the mere price of fuel. Because it utilizes ultra-cheap liquid oxygen and methane, shipping cargo to space will become more economical than flying cargo across Earth's oceans on an airplane. "The fundamental breakthrough of Starship is that it will be the first orbital rocket that is fully reusable... And the propellant we use for Starship is liquid oxygen and liquid methane, which is the cheapest propellant you could possibly get... which means that you should be able to actually send cargo to space for less than the cost of cargo on an airplane going on a trans-oceanic trip." 🔄 Starship V4 targets hourly launch cadences SpaceX's engineering pipeline is aiming for staggering operational frequencies and massive payloads. While Starship V3 targets 100 tons to orbit, the upcoming V4 variant is designed to carry over 200 tons and launch on an hourly schedule. "Because Starship V3 is aiming to do 100 tons to orbit with full reusability. And then Starship V4 we're aiming for over 200 tons per mission. And then being able to launch every hour." ☁️ Orbital data centers are entirely weather-proof Space-based AI data centers are highly practical because they are simpler to construct than communication satellites. Data is beamed via lasers between satellites, and then beamed to the ground using cloud-penetrating radio frequencies that completely bypass bad weather. "The AI data center would be much simpler by comparison. Because it's really just solar power plus radiator... The connection would happen no matter what the weather is. Because once you connect via the lasers to the Starlink communication constellation, the Starlink communication to the ground uses frequencies that are cloud penetrating." 🇺🇸 The U.S. faces a catastrophic "Zero Memory Fab" crisis A major vulnerability in domestic tech infrastructure is that the U.S. currently manufactures zero high-volume computer memory chips. Even with new facilities arriving online between 2028 and 2030, domestic supply will not match the exponential requirements of AI, which is why Musk is aggressively building the Terafab. "there's not a single high volume computer memory fab in America right now. Zero. There's one being built in Idaho by Micron. But that will not reach volume production until I believe 2028. And there's something being built in New York, but they are in, I think, 29 and 30. And this is a tiny fraction of the memory that's needed... That's why we need to do the Terafab." 🧠 SpaceX will offer proprietary AI chips and software While the orbital data center network will remain an open marketplace capable of running third-party hardware like NVIDIA GPUs, Google TPUs, or Amazon Trainium, SpaceX plans to deploy its own in-house AI chips and software stack in the near future. "So if NVIDIA GPUs can be put on it, Google TPUs can be put on it, Amazon Trainium or any other chips that you want to put on, can be put on. We'll also offer our chips in the future and I think we also want to offer our software, our AI software as well in the future." 🛡️ Starshield handles critical national intelligence Musk emphasizes his deeply pro-American stance, highlighting SpaceX's specialized Starshield division as a crucial backbone for the U.S. military and national intelligence agencies. "We have a division called Starshield which provides military communications. And you know, there's some other stuff that's kind of classified, I guess. We can't be talking about that. But we are helping the Department of War and intelligence part of the government. We're a vital element of that." 👥 Executive retention fuels the mission The core leadership bench at SpaceX is defined by extreme longevity, driven by a deep collective belief in turning science fiction into reality. Top executives like Gwynne Shotwell have remained with Musk for over two decades. "I guess Gwynne was, I think, around the seventh person to join the company. And that was 2002. It's just went to like 24 years. And generally the senior executives at the company, you have a very long tenure. I think Brent Johnson's been, you see, over 15 years... because people really believe in the mission, I think they want to stay and they want to keep building it." ❤️ Character overrides IQ in leadership Reflecting on how he has evolved over 20 years, Musk notes that he has become significantly more laid back. He has also learned that a candidate's moral character and heart are just as vital to a company's success as raw intellectual horsepower. "Well, I think I'm probably more chill than I used to be... And one of the things I've found over time... is that like in terms of like recruiting people to the company and having people work with the company, like their individual abilities and their intellectual capabilities matter a lot, but it also matters if they have a good heart. It's not just about whether somebody has a certain IQ or whatever, but just are they like a good person, that matters a lot."

Ming

60,910 görüntüleme • 3 ay önce