Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only... 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.show more

Varun

35,570 subscribers

37,555 Aufrufe • vor 4 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!

Andrew Ng

100,779 Aufrufe • vor 4 Monaten

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

MTS

44,057 Aufrufe • vor 2 Monaten

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,678 Aufrufe • vor 1 Monat

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Fireside Alpha

281,474 Aufrufe • vor 16 Tagen

I pay Claude $20 a month. Most $TAO holders do too. There is a stack you can build in 15 minutes that fixes that completely. It runs on Bittensor. It costs $10. You do not write a single line of code. Here is how every AI chat product actually works under the hood. Three layers. Always three. The model. The brain. GPT, Claude, DeepSeek, Kimi, GLM. The inference layer. The GPU that runs the model when you hit send. The interface. The chat box you actually look at. ChatGPT and Claude bundle all three and hand you the result. You cannot change the model. You cannot change the inference. The interface is non-negotiable. Every prompt you type goes to a server run by a private company whose terms of service can quietly change next month. The anti-ChatGPT move is to pick each layer yourself. This is where $TAO comes in. Chutes is Subnet 64 on Bittensor. It is the inference layer. Open source models like DeepSeek, Kimi, GLM, and Llama get served by a global network of miner-operated GPUs. Validators score the output quality. The best inference wins the emissions. You hit send. A miner somewhere runs your prompt. You get the answer back. The TAO you hold is in part paying for the GPU you just used. The basic stack is one URL. chutes. ai/chat No account. No API key. No setup. Switch models mid-conversation. Web search built in. Image generation. File uploads. Free. The advanced stack is Chutes plus TypingMind. One-time license. No recurring fee. Plugins, agents, custom personas, a prompt library you build over months. Full model switching between Chutes, OpenAI, and Anthropic from the same window. Total cost: $10 a month to Chutes for inference. That $10 buys you $50 in actual usage. But here is the signal most people missed inside this story. Chutes ran a free tier until February. Then they killed it. Then they raised the minimum to $10 in May. Most people saw that as bad news. It is the opposite. Free things on the internet do not last. Real products do. Chutes is becoming a real product. A subnet that generates actual revenue from actual users paying actual money for actual AI inference. That is what $43 million in Q1 network revenue looks like at the individual subnet level. And there is one more thing ChatGPT and Claude cannot offer that Chutes already has. Trusted Execution Environments. Your prompt gets encrypted on your device, shipped to a confidential compute GPU, and the lock only breaks inside the chip. The miner running the model physically cannot read your prompt. ChatGPT cannot promise that. Claude cannot promise that. Bittensor already built it. You are holding a network where the subnets are generating real revenue, shipping real privacy infrastructure, and replacing $20 a month centralised subscriptions with $10 a month decentralised inference. The people who use the product always understand the investment better than the people who only watch the price.

I pay Claude $20 a month. Most $TAO holders do too. There is a stack you can build in 15 minutes that fixes that completely. It runs on Bittensor. It costs $10. You do not write a single line of code. Here is how every AI chat product actually works under the hood. Three layers. Always three. The model. The brain. GPT, Claude, DeepSeek, Kimi, GLM. The inference layer. The GPU that runs the model when you hit send. The interface. The chat box you actually look at. ChatGPT and Claude bundle all three and hand you the result. You cannot change the model. You cannot change the inference. The interface is non-negotiable. Every prompt you type goes to a server run by a private company whose terms of service can quietly change next month. The anti-ChatGPT move is to pick each layer yourself. This is where $TAO comes in. Chutes is Subnet 64 on Bittensor. It is the inference layer. Open source models like DeepSeek, Kimi, GLM, and Llama get served by a global network of miner-operated GPUs. Validators score the output quality. The best inference wins the emissions. You hit send. A miner somewhere runs your prompt. You get the answer back. The TAO you hold is in part paying for the GPU you just used. The basic stack is one URL. chutes. ai/chat No account. No API key. No setup. Switch models mid-conversation. Web search built in. Image generation. File uploads. Free. The advanced stack is Chutes plus TypingMind. One-time license. No recurring fee. Plugins, agents, custom personas, a prompt library you build over months. Full model switching between Chutes, OpenAI, and Anthropic from the same window. Total cost: $10 a month to Chutes for inference. That $10 buys you $50 in actual usage. But here is the signal most people missed inside this story. Chutes ran a free tier until February. Then they killed it. Then they raised the minimum to $10 in May. Most people saw that as bad news. It is the opposite. Free things on the internet do not last. Real products do. Chutes is becoming a real product. A subnet that generates actual revenue from actual users paying actual money for actual AI inference. That is what $43 million in Q1 network revenue looks like at the individual subnet level. And there is one more thing ChatGPT and Claude cannot offer that Chutes already has. Trusted Execution Environments. Your prompt gets encrypted on your device, shipped to a confidential compute GPU, and the lock only breaks inside the chip. The miner running the model physically cannot read your prompt. ChatGPT cannot promise that. Claude cannot promise that. Bittensor already built it. You are holding a network where the subnets are generating real revenue, shipping real privacy infrastructure, and replacing $20 a month centralised subscriptions with $10 a month decentralised inference. The people who use the product always understand the investment better than the people who only watch the price.

2xnmore

27,088 Aufrufe • vor 2 Monaten

$70,000 Phones, One AI Agent — The World's Largest Edge AI Fleet Runs on Hermes We turned 70,000 phones into a shared AI compute network. Any device owner contributes idle compute. Any developer taps distributed inference at a fraction of cloud cost. Not a concept. Not a whitepaper. 70K devices online today. The problem: orchestrating a shared network of heterogeneous edge devices — different chipsets, different memory, different thermal profiles, different owners — is a coordination nightmare no human team can handle manually. So we gave the network a brain: Nous Research Hermes Agent. Hermes connects to 16 MCP servers and runs 24/7: 🔬 Research Loop — Tracks every breakthrough in on-device inference: quantization (GPTQ/AWQ/GGUF), speculative decoding on mobile SoCs, federated learning protocols. Auto-imports papers into NotebookLM. 36 research topics, zero manual curation. 🌐 Network Intelligence — Monitors device availability, compute capacity, and workload distribution across the shared fleet. Surfaces bottlenecks before they cascade. 🧬 Tech Tree Optimizer — Maps the full optimization frontier: from KV-cache compression to on-device LoRA to peer-to-peer model sharding. Hermes autonomously identifies which research paths unlock the most network-wide throughput gains. The result: a self-improving shared compute network. Research compounds daily. The fleet gets smarter without human intervention. Cloud AI scales with money. We scale with people. #HermesHackathon Teknium 🪽 Delphi Digital Tommy$

70,000 Phones, One AI Agent — The World's Largest Edge AI Fleet Runs on Hermes We turned 70,000 phones into a shared AI compute network. Any device owner contributes idle compute. Any developer taps distributed inference at a fraction of cloud cost. Not a concept. Not a whitepaper. 70K devices online today. The problem: orchestrating a shared network of heterogeneous edge devices — different chipsets, different memory, different thermal profiles, different owners — is a coordination nightmare no human team can handle manually. So we gave the network a brain: Nous Research Hermes Agent. Hermes connects to 16 MCP servers and runs 24/7: 🔬 Research Loop — Tracks every breakthrough in on-device inference: quantization (GPTQ/AWQ/GGUF), speculative decoding on mobile SoCs, federated learning protocols. Auto-imports papers into NotebookLM. 36 research topics, zero manual curation. 🌐 Network Intelligence — Monitors device availability, compute capacity, and workload distribution across the shared fleet. Surfaces bottlenecks before they cascade. 🧬 Tech Tree Optimizer — Maps the full optimization frontier: from KV-cache compression to on-device LoRA to peer-to-peer model sharding. Hermes autonomously identifies which research paths unlock the most network-wide throughput gains. The result: a self-improving shared compute network. Research compounds daily. The fleet gets smarter without human intervention. Cloud AI scales with money. We scale with people. #HermesHackathon Teknium 🪽 Delphi Digital Tommy

Oyster Republic 🦪📲🦞👓

20,721 Aufrufe • vor 5 Monaten

Gavin Baker (Gavin Baker) says the disaggregation of inference can extend GPU useful lives from 3-4 years to 10-15. That may single-handedly save private credit and reduce the financing rates for GPUs, which will drive demand and help finance the build-out. "The disaggregation of prefill and inference is going to be amazing for the useful lives of GPU and may single-handedly save private credit. Private credit is in pain from these SaaS loans. But there's a lot of private credit in GPUs too. They were underwriting that to 3-4. The disaggregation of inference means that these GPUs are going to have 10 or 15-year lives. The AI skeptics are like, "Oh, these companies are all cooking their books. The useful life of a GPU is only a year or two. The useful life of a CPU is only four years because the rapid technological change." No. What rapid technological change has done with the disaggregation of prefill and inference is you can put a Cerebras system or Groq LPUs effectively in front of a Hopper or even an Ampere, use that Hopper and Ampere for prefill, and extend the useful life of that GPU until it melts. This is going to be really good for the whole private credit industry. It's gonna help finance the AI build-out. Because if you can start to finance GPUs at 5% or 6% instead of – I think CoreWeave's lowest financing was low sevens – that actually mathematically changes the cost to finance this build-out."

Gavin Baker (Gavin Baker) says the disaggregation of inference can extend GPU useful lives from 3-4 years to 10-15. That may single-handedly save private credit and reduce the financing rates for GPUs, which will drive demand and help finance the build-out. "The disaggregation of prefill and inference is going to be amazing for the useful lives of GPU and may single-handedly save private credit. Private credit is in pain from these SaaS loans. But there's a lot of private credit in GPUs too. They were underwriting that to 3-4. The disaggregation of inference means that these GPUs are going to have 10 or 15-year lives. The AI skeptics are like, "Oh, these companies are all cooking their books. The useful life of a GPU is only a year or two. The useful life of a CPU is only four years because the rapid technological change." No. What rapid technological change has done with the disaggregation of prefill and inference is you can put a Cerebras system or Groq LPUs effectively in front of a Hopper or even an Ampere, use that Hopper and Ampere for prefill, and extend the useful life of that GPU until it melts. This is going to be really good for the whole private credit industry. It's gonna help finance the AI build-out. Because if you can start to finance GPUs at 5% or 6% instead of – I think CoreWeave's lowest financing was low sevens – that actually mathematically changes the cost to finance this build-out."

Invest Like the Best

211,336 Aufrufe • vor 2 Monaten

If intelligence is the log of compute… it starts with a lot of compute! And that’s why we’re scaling our GPU fleet faster than anyone else. Just last year, we added over 2 gigawatts of new capacity – roughly the output of 2 nuclear power plants. And today we’re going further, announcing the world's most powerful AI datacenter, located in southeastern Wisconsin. Fairwater is a seamless cluster of hundreds of thousands of NVIDIA GB200s, connected by enough fiber to circle the Earth 4.5 times. It will deliver 10x the performance of the world’s fastest supercomputer today, enabling AI training and inference workloads at a level never before seen. For AI training workloads, you need compute at exponential scale. That’s why we designed the datacenter, GPU fleet, and network together as one integrated system. This ensures a single job can run from day 1 at exponential scale across thousands of GPUs. Fairwater uses a liquid-cooled closed-loop system for cooling GPUs that requires zero water for operations after construction. And we’re matching all of the energy that is consumed with renewable sources. And of course, it is just one of several similar sites we’re lighting up across our 70+ regions. We have multiple identical Fairwater datacenters under construction in other locations across the US, in addition to our AI infrastructure already deployed in over 100 datacenters around the world, powering model training, test-time compute, RL tuning, and real-time inference at global scale. Too often during times like this, people go with the current and only later wonder, how did we get here? With Fairwater, we're charting a new path: doing the hard engineering work, bringing compute, network, and storage into one highly scaled cluster, and designing closed-loop energy systems to meet real-world computing needs. And partnering with local communities to ensure it's thoughtfully done in a way that is sustainable, creates new jobs, and expands opportunity. We are thrilled to see this take hold in Wisconsin, and we are just getting started.

If intelligence is the log of compute… it starts with a lot of compute! And that’s why we’re scaling our GPU fleet faster than anyone else. Just last year, we added over 2 gigawatts of new capacity – roughly the output of 2 nuclear power plants. And today we’re going further, announcing the world's most powerful AI datacenter, located in southeastern Wisconsin. Fairwater is a seamless cluster of hundreds of thousands of NVIDIA GB200s, connected by enough fiber to circle the Earth 4.5 times. It will deliver 10x the performance of the world’s fastest supercomputer today, enabling AI training and inference workloads at a level never before seen. For AI training workloads, you need compute at exponential scale. That’s why we designed the datacenter, GPU fleet, and network together as one integrated system. This ensures a single job can run from day 1 at exponential scale across thousands of GPUs. Fairwater uses a liquid-cooled closed-loop system for cooling GPUs that requires zero water for operations after construction. And we’re matching all of the energy that is consumed with renewable sources. And of course, it is just one of several similar sites we’re lighting up across our 70+ regions. We have multiple identical Fairwater datacenters under construction in other locations across the US, in addition to our AI infrastructure already deployed in over 100 datacenters around the world, powering model training, test-time compute, RL tuning, and real-time inference at global scale. Too often during times like this, people go with the current and only later wonder, how did we get here? With Fairwater, we're charting a new path: doing the hard engineering work, bringing compute, network, and storage into one highly scaled cluster, and designing closed-loop energy systems to meet real-world computing needs. And partnering with local communities to ensure it's thoughtfully done in a way that is sustainable, creates new jobs, and expands opportunity. We are thrilled to see this take hold in Wisconsin, and we are just getting started.

Satya Nadella

2,022,693 Aufrufe • vor 10 Monaten

Chamath said AI is not like the internet. Every new user costs real money. And the infrastructure making it possible was built by everyone. His argument was the clearest case for government ownership of AI labs I have ever heard. And it had nothing to do with Bernie Sanders. Start with the internet comparison. Google and Facebook became the most profitable companies in human history because of one number. The marginal cost of adding a new user was effectively zero. One more search query cost Google nothing. One more Facebook profile cost Meta nothing. They could serve a billion people and the incremental cost of that billion person was rounding error. That is the money printer. Infinite scale at zero marginal cost. AI breaks that model completely. Every single user taxes a GPU. Every query costs electricity. Every response requires memory and compute. The marginal cost of AI is real, significant, and does not disappear at scale. You cannot print money the same way. Then Chamath made the point that landed hardest. The infrastructure these companies depend on, the power grid, the land, the data centers, the permitting, the national security apparatus that protects their chips from being stolen, none of that was built by Anthropic or OpenAI. It was built by the public. By taxpayers. By decades of government investment in the physical and legal foundation these companies are now running on. He compared it to the interstate highway system. If the federal government built the roads and two companies transported all the goods on them, a logical question at that point would be how much of that should I own? You are riding on my rails. His conclusion was direct. If he were running a sovereign wealth fund and had the negotiating leverage of the US government, he would own 75% of these companies when he was done. The internet had zero marginal cost. That is why the founders captured almost all of the value. AI has real marginal cost and runs on public infrastructure. That changes who has a claim on what gets built. WATCH THE FULL PODCAST ON The All-In Podcast

Ihtesham Ali

79,066 Aufrufe • vor 1 Monat

$Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.$

Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.

Milk Road AI

27,908 Aufrufe • vor 1 Monat

Destra Edge — $DSYNC From Idea to Reality Yesterday, we introduced Destra Edge. Today, we show you what it means. For too long, AI has lived in data centers. Expensive. Centralized. Controlled by a few. Meanwhile, billions of smartphones sit idle. Powerful GPUs. Doing nothing. That doesn’t make sense. So we’re building something different. Destra Edge turns everyday smartphones into a global AI inference network. No servers. No cloud monopolies. Just real compute at the edge. This video shows the system taking shape — how inference is distributed, how it runs on-device, and how results come together without central control. This is not finished. And that’s the point. You’re watching it being built. Layer by layer. In public. The magic is in the scale. More phones means more power. More power means stronger AI. The network grows simply because people join. No passive mining. No fake activity. Only real GPU inference. Measured. Verified. Rewarded. AI shouldn’t live in warehouses. It should live in your pocket. This is edge-level decentralization. This is permissionless inference. This is Destra Network.

Destra Edge — $DSYNC From Idea to Reality Yesterday, we introduced Destra Edge. Today, we show you what it means. For too long, AI has lived in data centers. Expensive. Centralized. Controlled by a few. Meanwhile, billions of smartphones sit idle. Powerful GPUs. Doing nothing. That doesn’t make sense. So we’re building something different. Destra Edge turns everyday smartphones into a global AI inference network. No servers. No cloud monopolies. Just real compute at the edge. This video shows the system taking shape — how inference is distributed, how it runs on-device, and how results come together without central control. This is not finished. And that’s the point. You’re watching it being built. Layer by layer. In public. The magic is in the scale. More phones means more power. More power means stronger AI. The network grows simply because people join. No passive mining. No fake activity. Only real GPU inference. Measured. Verified. Rewarded. AI shouldn’t live in warehouses. It should live in your pocket. This is edge-level decentralization. This is permissionless inference. This is Destra Network.

Destra Network

20,156 Aufrufe • vor 7 Monaten

after turboquant and qwen3.5-35b-a3b, i got curious: how realistic is it to use kv cache as a document store today? to have vectorless, RAG-less search. so i prefilled 258K out of 262K context window on L4 (a budget GPU popular in prod). ~99% of the slot is pre-computed and stored, users load it on the fly in ~1s. system prompt + query append to the end, generation takes ~3K tokens, enough for search. at 99% fill rate, decoding runs ~20 tps on L4. i prepared some ego datasets (jina papers, which i know best), plus popular novels in chinese and english. the results are actually pretty good. some hallucination, but most answers are solid and well-grounded. what's more interesting is the cost: ~$0.26/h on L4 spot. single LLM. no vector database, no embedding model, no workflow/pipeline engineering. using kv cache as document store is nothing new, like the old CAG paper. but with quantized kv cache and modern attention (hybrid SSM-attention, GQA, MQA, MLA), the economics are changing fast. if we solve cold-prefill speed and decoding speed, and budget GPU costs keep dropping, the future of search could be vectorless. radical, but possible.

after turboquant and qwen3.5-35b-a3b, i got curious: how realistic is it to use kv cache as a document store today? to have vectorless, RAG-less search. so i prefilled 258K out of 262K context window on L4 (a budget GPU popular in prod). ~99% of the slot is pre-computed and stored, users load it on the fly in ~1s. system prompt + query append to the end, generation takes ~3K tokens, enough for search. at 99% fill rate, decoding runs ~20 tps on L4. i prepared some ego datasets (jina papers, which i know best), plus popular novels in chinese and english. the results are actually pretty good. some hallucination, but most answers are solid and well-grounded. what's more interesting is the cost: ~$0.26/h on L4 spot. single LLM. no vector database, no embedding model, no workflow/pipeline engineering. using kv cache as document store is nothing new, like the old CAG paper. but with quantized kv cache and modern attention (hybrid SSM-attention, GQA, MQA, MLA), the economics are changing fast. if we solve cold-prefill speed and decoding speed, and budget GPU costs keep dropping, the future of search could be vectorless. radical, but possible.

Han Xiao

42,430 Aufrufe • vor 4 Monaten

Today we announced our new Fairwater datacenter in Atlanta, connected with our first Fairwater site in Wisconsin and our broader Azure footprint to create the world’s first AI superfactory. Fairwater exemplifies our vision for a fungible fleet: infra that can serve any workload, anywhere, on fit-for-purpose accelerators and network paths, with maximum performance and efficiency. AI workloads have evolved beyond large-scale pre-training. Today, they encompass fine-tuning, reinforcement learning (RL), synthetic data generation, evaluation pipelines, and more. Fairwater is built to support this full lifecycle: Max density: Fairwater’s two-story design and liquid cooling system lets us place racks in three dimensions and pack them with GPUs as densely as possible, minimizing cable runs and improving latency and effective bandwidth. Fleet: Each Fairwater DC can integrate hundreds of thousands of the latest NVIDIA GPUs into a single coherent cluster. This provides flexible infra that can support the full spectrum of workloads, and ensure no GPU is left unnecessarily idle. And that’s on top of the more than 100,000 GB300s coming online this quarter alone for inference across the rest of our fleet. For us, it’s all about turning every gigawatt into the maximum number of useful tokens. Not every GW is created equal! Planet-scale: Every Fairwater DC will connect through our continent-spanning AI WAN to prior generations of AI supercomputers, forming a truly fungible pool of compute. This enables developers to scale beyond the capacity of a single site and dynamically land workloads on the right infra for their needs. Together, these innovations let us bring together different generations of silicon and AI systems across DCs and geos into a single elastic system that scales seamlessly across training and inference workloads And this elastic AI capacity is all available alongside all the other cloud services (compute, storage, databases, app services) that AI agents and workloads need. This is what we mean when we talk about building a fungible fleet – a single, unified platform that pushes the limits of performance per watt and per dollar. Read more:

Today we announced our new Fairwater datacenter in Atlanta, connected with our first Fairwater site in Wisconsin and our broader Azure footprint to create the world’s first AI superfactory. Fairwater exemplifies our vision for a fungible fleet: infra that can serve any workload, anywhere, on fit-for-purpose accelerators and network paths, with maximum performance and efficiency. AI workloads have evolved beyond large-scale pre-training. Today, they encompass fine-tuning, reinforcement learning (RL), synthetic data generation, evaluation pipelines, and more. Fairwater is built to support this full lifecycle: Max density: Fairwater’s two-story design and liquid cooling system lets us place racks in three dimensions and pack them with GPUs as densely as possible, minimizing cable runs and improving latency and effective bandwidth. Fleet: Each Fairwater DC can integrate hundreds of thousands of the latest NVIDIA GPUs into a single coherent cluster. This provides flexible infra that can support the full spectrum of workloads, and ensure no GPU is left unnecessarily idle. And that’s on top of the more than 100,000 GB300s coming online this quarter alone for inference across the rest of our fleet. For us, it’s all about turning every gigawatt into the maximum number of useful tokens. Not every GW is created equal! Planet-scale: Every Fairwater DC will connect through our continent-spanning AI WAN to prior generations of AI supercomputers, forming a truly fungible pool of compute. This enables developers to scale beyond the capacity of a single site and dynamically land workloads on the right infra for their needs. Together, these innovations let us bring together different generations of silicon and AI systems across DCs and geos into a single elastic system that scales seamlessly across training and inference workloads And this elastic AI capacity is all available alongside all the other cloud services (compute, storage, databases, app services) that AI agents and workloads need. This is what we mean when we talk about building a fungible fleet – a single, unified platform that pushes the limits of performance per watt and per dollar. Read more:

Satya Nadella

907,893 Aufrufe • vor 9 Monaten

Gavin Baker says the piece missing from the compute debate is SRAM accelerators, and disaggregating inference across three chips lifts AI's ROI. "I do think something that is missing from all of this conversation about compute is what is going to happen when you put these SRAM-based accelerators that are not constrained by HBM DRAM and are often made on older nodes that are not competing with the latest greatest GPUs". "When you disaggregate inference, people talk about pre-fill and decode, but decode is two parts, attention and feed-forward network." "The ultimate holy grail is if you could do pre-fill on one chip that probably doesn't have HBM DRAM, do the attention on a super high-powered chip with HBM DRAM, and then do the feed-forward network on one of these SRAM chips." "You just can't beat SRAM in particular for that feed-forward network, and you just almost can't, no matter how much you try and get the ratio of compute to HBM DRAM to SRAM on the chip correct, the workloads are always changing, and there's different workloads." "Being able to disaggregate into these three parts, I think this is going to be really, really positive for the ROI on AI."

Gavin Baker says the piece missing from the compute debate is SRAM accelerators, and disaggregating inference across three chips lifts AI's ROI. "I do think something that is missing from all of this conversation about compute is what is going to happen when you put these SRAM-based accelerators that are not constrained by HBM DRAM and are often made on older nodes that are not competing with the latest greatest GPUs". "When you disaggregate inference, people talk about pre-fill and decode, but decode is two parts, attention and feed-forward network." "The ultimate holy grail is if you could do pre-fill on one chip that probably doesn't have HBM DRAM, do the attention on a super high-powered chip with HBM DRAM, and then do the feed-forward network on one of these SRAM chips." "You just can't beat SRAM in particular for that feed-forward network, and you just almost can't, no matter how much you try and get the ratio of compute to HBM DRAM to SRAM on the chip correct, the workloads are always changing, and there's different workloads." "Being able to disaggregate into these three parts, I think this is going to be really, really positive for the ROI on AI."

Fireside Alpha

15,501 Aufrufe • vor 7 Tagen

The value of the work we're doing at Optimum is encapsulated quite well by the phrase "speed is money". In modern markets there are real economic advantages to latency reduction. This is nothing new. Wall Street firms have long been optimizing on latency, primarily through colocation and top of the line hardware. However, when it comes to decentralized systems, expensive hardware and geographic concentration are antithetical to their purpose. Therefore we should optimize decentralized network latency through software, which I'm thrilled about because it's exactly what I've spent the better part of the past 2 decades working on with Random Linear Network Coding. Now let’s talk about networking economics, the relationship between speed and money. First, it's important to note that users will only pay for low latency if it can be consistently guaranteed. Second, you can only make that latency guarantee for a certain number of users. This is a universal law of networking. We can model this relationship on a delay curve, shown below. The delay curve is determined by the utilization rate of the network, meaning how much traffic is flowing through the network divided by the network's throughput. As you approach a level of traffic equal to the available throughput, latency trends infinitely higher. On this delay curve we can impose some utility thresholds. These thresholds are the levels of latency which are important to different groups of users because of how that latency guarantee improves their economic outcomes. Finding the point on the curve where each threshold intersects will tell us what level of traffic we can guarantee that level of latency for. Essentially, there exists a finite supply of speed on a network and the highest utility users of that speed are willing to pay more for it. I like to think of this similarly to expedited shipping options on Amazon. This is why we say speed is money, and why we can create a Latency Marketplace. The only way to increase the supply of speed is to fundamentally increase network throughput. This is what we work on at Optimum by using Random Linear Network Coding. The same relationship between traffic and throughput still applies, but now the delay curve is shifted out further to the right. Now more traffic can be processed at the same latency, or the same traffic can be processed at a lower latency. More speed available to the network. More value unlocked for the network’s users. Crucially, that value is no longer only reserved for those who can afford to sit closest to the machine. Expanding the supply of speed widens who can reach each latency threshold, keeping the network's advantage decentralized rather than concentrated in the hands of a few. When nodes join Optimum and participate, they reap the benefits, but they also add to the capacity. Rather than vying against each other in a zero-sum game, nodes help themselves and others.

The value of the work we're doing at Optimum is encapsulated quite well by the phrase "speed is money". In modern markets there are real economic advantages to latency reduction. This is nothing new. Wall Street firms have long been optimizing on latency, primarily through colocation and top of the line hardware. However, when it comes to decentralized systems, expensive hardware and geographic concentration are antithetical to their purpose. Therefore we should optimize decentralized network latency through software, which I'm thrilled about because it's exactly what I've spent the better part of the past 2 decades working on with Random Linear Network Coding. Now let’s talk about networking economics, the relationship between speed and money. First, it's important to note that users will only pay for low latency if it can be consistently guaranteed. Second, you can only make that latency guarantee for a certain number of users. This is a universal law of networking. We can model this relationship on a delay curve, shown below. The delay curve is determined by the utilization rate of the network, meaning how much traffic is flowing through the network divided by the network's throughput. As you approach a level of traffic equal to the available throughput, latency trends infinitely higher. On this delay curve we can impose some utility thresholds. These thresholds are the levels of latency which are important to different groups of users because of how that latency guarantee improves their economic outcomes. Finding the point on the curve where each threshold intersects will tell us what level of traffic we can guarantee that level of latency for. Essentially, there exists a finite supply of speed on a network and the highest utility users of that speed are willing to pay more for it. I like to think of this similarly to expedited shipping options on Amazon. This is why we say speed is money, and why we can create a Latency Marketplace. The only way to increase the supply of speed is to fundamentally increase network throughput. This is what we work on at Optimum by using Random Linear Network Coding. The same relationship between traffic and throughput still applies, but now the delay curve is shifted out further to the right. Now more traffic can be processed at the same latency, or the same traffic can be processed at a lower latency. More speed available to the network. More value unlocked for the network’s users. Crucially, that value is no longer only reserved for those who can afford to sit closest to the machine. Expanding the supply of speed widens who can reach each latency threshold, keeping the network's advantage decentralized rather than concentrated in the hands of a few. When nodes join Optimum and participate, they reap the benefits, but they also add to the capacity. Rather than vying against each other in a zero-sum game, nodes help themselves and others.

Muriel Medard

44,225 Aufrufe • vor 1 Monat

AI is the first technology in history where more customers makes you POORER. Every tech company in history got cheaper as it scaled. More users meant lower costs per user. That's the entire model. That's why Microsoft prints money. That's why Google prints money. That's why Meta prints money. Software has near-zero marginal cost. Build it once. Sell it a billion times. The 100 millionth user costs basically nothing to serve. This is the single most important rule in tech economics. But AI completely broke it. Every single query costs real compute. Every interaction burns real electricity. Every response depreciates real hardware. There is no "build once, sell forever." There is only "burn money every time someone asks a question." And the numbers prove it: OpenAI hit $20 billion in annualized revenue. Losses? $14 billion. For every dollar they earn, they spend $1.69 delivering it. Their losses TRIPLED as their revenue grew. Not because they're bad at business, but simply because the model itself is broken. Anthropic crossed $30 billion in annualized revenue. Still burning billions. Still not profitable. Still raising tens of billions just to keep the lights on. xAI is burning $1 billion every single month. Perplexity spent 164% of its revenue on compute costs from AWS, They literally spent more on running the AI than they made from selling it. This is not how technology is supposed to work. Google once estimated that adding AI to every search query would require 500,000 A100 servers. The cost of answering a single AI query is 10x MORE than a traditional search result. Traditional software: Serving 1 million users costs roughly the same as serving 100,000. The marginal cost is basically zero. AI: Serving 1 million users can cost 10 times what 100,000 costs. Every new user is a new expense. Every new query is a new dollar burned. This is reverse economics. The more successful you become, the faster you die. And nobody in the industry wants to talk about it because the entire narrative depends on you believing AI companies work like software companies. But they don't. They NEVER will. Software scales to infinity. AI scales to bankruptcy. HSBC ran the numbers on OpenAI specifically. Their conclusion: Even after every funding round, every investment, every deal, OpenAI still faces a $207 BILLION shortfall to reach profitability. The industry response has been to raise prices. ChatGPT went from free to $20 to $200 for the Pro plan. And it's still not enough because the cost of running these models grows FASTER than any price increase consumers will accept. Meanwhile 966 AI startups died in 2024. A 25.6% jump from the year before. AI startups burn cash twice as fast as non-AI tech companies. And the ones building on TOP of OpenAI and Anthropic are in even worse shape. Every wrapper app. Every "AI-powered" SaaS tool. Every startup whose entire product is someone else's model with a different skin on it. They're all margin-negative. Every single one. And these are the companies about to IPO. SpaceX, OpenAI, Anthropic, and Cerebras. $240 billion in combined raises planned for 2026. They're asking you to invest in an industry where the fundamental unit economics don't work. Where the MORE customers you get, the MORE money you lose. Where no company has figured out how to make the math positive. The dot-com bubble had the same pitch: "Revenue is growing. Profitability comes later." For most of them, later never came. The question isn't whether AI will change the world. It will. The question is whether it can do it without going broke first. And right now, every single number literally says no. How can they become profitable?

AI is the first technology in history where more customers makes you POORER. Every tech company in history got cheaper as it scaled. More users meant lower costs per user. That's the entire model. That's why Microsoft prints money. That's why Google prints money. That's why Meta prints money. Software has near-zero marginal cost. Build it once. Sell it a billion times. The 100 millionth user costs basically nothing to serve. This is the single most important rule in tech economics. But AI completely broke it. Every single query costs real compute. Every interaction burns real electricity. Every response depreciates real hardware. There is no "build once, sell forever." There is only "burn money every time someone asks a question." And the numbers prove it: OpenAI hit $20 billion in annualized revenue. Losses? $14 billion. For every dollar they earn, they spend $1.69 delivering it. Their losses TRIPLED as their revenue grew. Not because they're bad at business, but simply because the model itself is broken. Anthropic crossed $30 billion in annualized revenue. Still burning billions. Still not profitable. Still raising tens of billions just to keep the lights on. xAI is burning $1 billion every single month. Perplexity spent 164% of its revenue on compute costs from AWS, They literally spent more on running the AI than they made from selling it. This is not how technology is supposed to work. Google once estimated that adding AI to every search query would require 500,000 A100 servers. The cost of answering a single AI query is 10x MORE than a traditional search result. Traditional software: Serving 1 million users costs roughly the same as serving 100,000. The marginal cost is basically zero. AI: Serving 1 million users can cost 10 times what 100,000 costs. Every new user is a new expense. Every new query is a new dollar burned. This is reverse economics. The more successful you become, the faster you die. And nobody in the industry wants to talk about it because the entire narrative depends on you believing AI companies work like software companies. But they don't. They NEVER will. Software scales to infinity. AI scales to bankruptcy. HSBC ran the numbers on OpenAI specifically. Their conclusion: Even after every funding round, every investment, every deal, OpenAI still faces a $207 BILLION shortfall to reach profitability. The industry response has been to raise prices. ChatGPT went from free to $20 to $200 for the Pro plan. And it's still not enough because the cost of running these models grows FASTER than any price increase consumers will accept. Meanwhile 966 AI startups died in 2024. A 25.6% jump from the year before. AI startups burn cash twice as fast as non-AI tech companies. And the ones building on TOP of OpenAI and Anthropic are in even worse shape. Every wrapper app. Every "AI-powered" SaaS tool. Every startup whose entire product is someone else's model with a different skin on it. They're all margin-negative. Every single one. And these are the companies about to IPO. SpaceX, OpenAI, Anthropic, and Cerebras. $240 billion in combined raises planned for 2026. They're asking you to invest in an industry where the fundamental unit economics don't work. Where the MORE customers you get, the MORE money you lose. Where no company has figured out how to make the math positive. The dot-com bubble had the same pitch: "Revenue is growing. Profitability comes later." For most of them, later never came. The question isn't whether AI will change the world. It will. The question is whether it can do it without going broke first. And right now, every single number literally says no. How can they become profitable?

Ricardo

167,414 Aufrufe • vor 3 Monaten

Lightchain AI – Decentralized Inference Demo 🍂🥧 + Blockchain Rewards In this video, we showcase how Lightchain AI’s decentralized inference engine (AIVM + PoI + Federated Learning) runs a real-world prompt “What is the best way to make pumpkin pie for thanksgiving dinner?” 💰 Blockchain Rewards in Action This demo doesn’t just show inference — it shows how inference providers are rewarded for their work. The provider that processed this prompt was automatically rewarded on the Lightchain testnet. 🍁 Why It Matters Unlike centralized AI, Lightchain AI distributes inference across a global network of participants. Nodes earn blockchain rewards for providing compute, ensuring a trustless, transparent, and economically sustainable ecosystem. With pay-to-prompt mechanics and DAO governance, Lightchain AI creates a fair AI economy for all. 🌐 Learn more at-

Lightchain AI – Decentralized Inference Demo 🍂🥧 + Blockchain Rewards In this video, we showcase how Lightchain AI’s decentralized inference engine (AIVM + PoI + Federated Learning) runs a real-world prompt “What is the best way to make pumpkin pie for thanksgiving dinner?” 💰 Blockchain Rewards in Action This demo doesn’t just show inference — it shows how inference providers are rewarded for their work. The provider that processed this prompt was automatically rewarded on the Lightchain testnet. 🍁 Why It Matters Unlike centralized AI, Lightchain AI distributes inference across a global network of participants. Nodes earn blockchain rewards for providing compute, ensuring a trustless, transparent, and economically sustainable ecosystem. With pay-to-prompt mechanics and DAO governance, Lightchain AI creates a fair AI economy for all. 🌐 Learn more at-

Lightchain AI

18,665 Aufrufe • vor 11 Monaten

"I used to be a bitcoiner. The transition to a new store of value only happens once every 3,000 years. That's the main prize -- just focus on that. But [security] is the criteria that ultimately convinced me to flip from Bitcoin to ETH." "I have a higher degree of certainty that Ethereum will be around longer [than Bitcoin]. The reason for that is because Bitcoin relies on proof-of-work, which is less efficient than proof-of-stake and doesn't scale with the value of the network. And as the block subsidy of Bitcoin halves every four years, it is increasingly becoming more and more reliant on transaction fees to fund the security budget paid to miners." "If you look at [Bitcoin's] security budget right now, about 0.6% of revenue to miners is transaction fees... The problem with that is if Bitcoin becomes 'digital gold', flips gold, and becomes a $30 trillion asset, but it only costs $10-20 billion to attack it, that's too asymmetric." "You want the security budget to scale with the market cap, similar to how countries spend a % of their GDP on defense. The more valuable something is, the more you need to spend to protect it." "Ethereum, with the Merge, migrated to proof-of-stake, which is fundamentally more secure because it's less reliant on transaction fees and it scales with the value of the network. If 1/3rd of ETH is staked and then you need 1/3rd of those ETH to censor the network, you're looking at roughly 10% of the total market cap as the cost to attack the network." "So if Ethereum flips Bitcoin and gold and becomes a $30 trillion asset, it'll cost ~$3 trillion to attack the Ethereum network versus Bitcoin at like $10 billion." "The other aspect here is that as AI hyperscalers invest more and more in AI, proof-of-work becomes increasingly vulnerable because the cost to attack the Bitcoin network is starting to look close to the quarterly CapEx these hyperscalers are spending on their data centers." Full interview on Bankless with Vivek Raman discussing the new Etherealize "Productive Money" report below.

"I used to be a bitcoiner. The transition to a new store of value only happens once every 3,000 years. That's the main prize -- just focus on that. But [security] is the criteria that ultimately convinced me to flip from Bitcoin to ETH." "I have a higher degree of certainty that Ethereum will be around longer [than Bitcoin]. The reason for that is because Bitcoin relies on proof-of-work, which is less efficient than proof-of-stake and doesn't scale with the value of the network. And as the block subsidy of Bitcoin halves every four years, it is increasingly becoming more and more reliant on transaction fees to fund the security budget paid to miners." "If you look at [Bitcoin's] security budget right now, about 0.6% of revenue to miners is transaction fees... The problem with that is if Bitcoin becomes 'digital gold', flips gold, and becomes a $30 trillion asset, but it only costs $10-20 billion to attack it, that's too asymmetric." "You want the security budget to scale with the market cap, similar to how countries spend a % of their GDP on defense. The more valuable something is, the more you need to spend to protect it." "Ethereum, with the Merge, migrated to proof-of-stake, which is fundamentally more secure because it's less reliant on transaction fees and it scales with the value of the network. If 1/3rd of ETH is staked and then you need 1/3rd of those ETH to censor the network, you're looking at roughly 10% of the total market cap as the cost to attack the network." "So if Ethereum flips Bitcoin and gold and becomes a $30 trillion asset, it'll cost ~$3 trillion to attack the Ethereum network versus Bitcoin at like $10 billion." "The other aspect here is that as AI hyperscalers invest more and more in AI, proof-of-work becomes increasingly vulnerable because the cost to attack the Bitcoin network is starting to look close to the quarterly CapEx these hyperscalers are spending on their data centers." Full interview on Bankless with Vivek Raman discussing the new Etherealize "Productive Money" report below.

Michael McGuiness

120,456 Aufrufe • vor 3 Monaten

someone just rebuilt claude code in pure python and made it ~230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 677 stars. basically nobody is looking yet bookmark this before it gone

someone just rebuilt claude code in pure python and made it ~230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 677 stars. basically nobody is looking yet bookmark this before it gone

Argona

51,369 Aufrufe • vor 1 Monat

Great explanation by Emad Mostaque, co-founder of Stability AI. "We’ll see the cost of Kimi K3 drop by 10 to 50 times, I think, over the next few months as it gets optimized. " Basically Kimi K3’s current inference cost is quite high, but that price reflects immature infrastructure, not a permanent technical limit. And that gap will not last long. US-based specialized infrastructure companies will optimize kernels, routing, quantization, batching, memory use, and serving systems around those models once the Kimi K3 weights are available. --- "Right now, it uses twice the number of tokens for the same task compared with GPT-5.6. Again, we’re going to see that cost drop because everyone and their dog is going to optimize the crap out of this. Fireworks has just raised funding at a $17 billion valuation, while others, such as Modal and Baseten, are valued at $10 billion. These are inference providers for open-source models. They’ve all raised around a billion dollars, which they’re now going to spend on optimizing the Chinese model, making it more efficient, and running it. American labs that handle the inference side of things are going to optimize the crap out of this. Therefore, we will see it catch up." ---- From "Peter H. Diamandis" YouTube channel, (full video link in comment)

Great explanation by Emad Mostaque, co-founder of Stability AI. "We’ll see the cost of Kimi K3 drop by 10 to 50 times, I think, over the next few months as it gets optimized. " Basically Kimi K3’s current inference cost is quite high, but that price reflects immature infrastructure, not a permanent technical limit. And that gap will not last long. US-based specialized infrastructure companies will optimize kernels, routing, quantization, batching, memory use, and serving systems around those models once the Kimi K3 weights are available. --- "Right now, it uses twice the number of tokens for the same task compared with GPT-5.6. Again, we’re going to see that cost drop because everyone and their dog is going to optimize the crap out of this. Fireworks has just raised funding at a $17 billion valuation, while others, such as Modal and Baseten, are valued at $10 billion. These are inference providers for open-source models. They’ve all raised around a billion dollars, which they’re now going to spend on optimizing the Chinese model, making it more efficient, and running it. American labs that handle the inference side of things are going to optimize the crap out of this. Therefore, we will see it catch up." ---- From "Peter H. Diamandis" YouTube channel, (full video link in comment)

Rohan Paul

82,007 Aufrufe • vor 22 Tagen