Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Kimi K3 is too large to run efficiently on a single accelerator so the workload must be distributed across many chips that exchange data at extremely high speeds. $CRDO and $ALAB benefit from the high speed connectivity inside AI clusters while $MRVL and $AVGO provide the networking, optical and... show more

Shay Boloor

447,872 subscribers

84,505 Aufrufe • vor 1 Monat •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

$AVGO helps hyperscalers build the custom chips and networking fabric needed to run massive AI systems. If $GOOGL, $META, OpenAI or Anthropic want to build their own inference chips then Broadcom is one of the first calls because it has the IP, packaging expertise and networking portfolio required to scale custom AI systems. Thats why Broadcom remains one of the strongest infrastructure positions in the AI economy.

$AVGO helps hyperscalers build the custom chips and networking fabric needed to run massive AI systems. If $GOOGL, $META, OpenAI or Anthropic want to build their own inference chips then Broadcom is one of the first calls because it has the IP, packaging expertise and networking portfolio required to scale custom AI systems. Thats why Broadcom remains one of the strongest infrastructure positions in the AI economy.

Shay Boloor

77,765 Aufrufe • vor 3 Monaten

Nvidia's next generation chips are about to make a handful of companies impossible to ignore and here is how you can benefit from this (Save this). Goldman projects networking content per AI compute system could rise from $315,000 today to $9.4 million in Nvidia's next generation, a 29x increase in what companies spend just wiring their chips together. Morgan Stanley, Citi and Goldman all agree on the same direction, projecting the total addressable market for AI networking to grow from around $11 to $15 billion today to $154 billion by 2028, roughly a 9x jump in just a few years. As AI clusters scale into the hundreds of thousands of GPUs, the chips themselves stop being the bottleneck and the connections between them become the limiting factor instead. Every GPU needs to talk to every other GPU almost instantly to keep a training run synchronized and that requires far more sophisticated switches, optical modules and cabling than today's networks can provide, which is exactly why Nvidia's next generation systems are expected to need dramatically more networking hardware per system just to keep up. Optical modules specifically are becoming the chokepoint. The global optical module total addressable market is projected to grow from roughly $6.7 billion in the first quarter of 2025 to nearly $69 billion by 2028, with 1.6T modules going from essentially zero market share today to capturing the majority of shipments by 2028 as speeds keep climbing. Nvidia sits at the center of this story since it designs the switches and networking silicon that go into every one of these systems, meaning that 29x jump in networking content per system flows straight into Nvidia's own revenue per AI factory, not just its GPU sales. Marvell is one of the biggest direct beneficiaries here, since it makes the custom networking chips and interconnect silicon that hyperscalers rely on to move data between GPUs at these speeds. Credo Technology plays a slightly different role, supplying the high speed connectivity chips that clean up signals moving through copper links, which becomes more valuable as networks push toward higher bandwidth with lower power draw. AAOI, or Applied Optoelectronics, sits closer to the physical optics layer, making the lasers and optical components that go into the transceivers carrying data across these networks, positioning it directly in the path of that jump from $6.7 billion to $69 billion in optical module demand. Coherent and Lumentum round out the optical side, both supplying lasers and photonic components that scale directly with 1.6T module adoption as it goes from near zero to the majority of shipments by 2028. Switch and connector makers benefit almost mechanically from this trend too, since every additional dollar of networking content per system has to physically pass through a switch, a cable, or a connector, meaning revenue scales with network complexity regardless of which chipmaker's GPUs sit inside the rack. Bullish on AI networking, make sure to follow Melvin for more semiconductor insights and check out the link below for more details.

Nvidia's next generation chips are about to make a handful of companies impossible to ignore and here is how you can benefit from this (Save this). Goldman projects networking content per AI compute system could rise from $315,000 today to $9.4 million in Nvidia's next generation, a 29x increase in what companies spend just wiring their chips together. Morgan Stanley, Citi and Goldman all agree on the same direction, projecting the total addressable market for AI networking to grow from around $11 to $15 billion today to $154 billion by 2028, roughly a 9x jump in just a few years. As AI clusters scale into the hundreds of thousands of GPUs, the chips themselves stop being the bottleneck and the connections between them become the limiting factor instead. Every GPU needs to talk to every other GPU almost instantly to keep a training run synchronized and that requires far more sophisticated switches, optical modules and cabling than today's networks can provide, which is exactly why Nvidia's next generation systems are expected to need dramatically more networking hardware per system just to keep up. Optical modules specifically are becoming the chokepoint. The global optical module total addressable market is projected to grow from roughly $6.7 billion in the first quarter of 2025 to nearly $69 billion by 2028, with 1.6T modules going from essentially zero market share today to capturing the majority of shipments by 2028 as speeds keep climbing. Nvidia sits at the center of this story since it designs the switches and networking silicon that go into every one of these systems, meaning that 29x jump in networking content per system flows straight into Nvidia's own revenue per AI factory, not just its GPU sales. Marvell is one of the biggest direct beneficiaries here, since it makes the custom networking chips and interconnect silicon that hyperscalers rely on to move data between GPUs at these speeds. Credo Technology plays a slightly different role, supplying the high speed connectivity chips that clean up signals moving through copper links, which becomes more valuable as networks push toward higher bandwidth with lower power draw. AAOI, or Applied Optoelectronics, sits closer to the physical optics layer, making the lasers and optical components that go into the transceivers carrying data across these networks, positioning it directly in the path of that jump from $6.7 billion to $69 billion in optical module demand. Coherent and Lumentum round out the optical side, both supplying lasers and photonic components that scale directly with 1.6T module adoption as it goes from near zero to the majority of shipments by 2028. Switch and connector makers benefit almost mechanically from this trend too, since every additional dollar of networking content per system has to physically pass through a switch, a cable, or a connector, meaning revenue scales with network complexity regardless of which chipmaker's GPUs sit inside the rack. Bullish on AI networking, make sure to follow Melvin for more semiconductor insights and check out the link below for more details.

Melvin

42,220 Aufrufe • vor 1 Monat

The optical networking supercycle is here! In this podcast, and Vikram Sekar go through all the tech jargon and explain what everything means. In just about 45 mins, you will know everything required to keep up with the next revolution in AI. Chapters 00:00 Introduction to AI and CPU Bottlenecks 03:00 The Rise of Silicon Photonics 06:01 Understanding Optical Networking and Data Centers 08:49 Scale Across: Connecting Data Centers 11:56 Scale Out: Optimizing Data Center Connectivity 14:53 Scale Up: The Future of GPU Connectivity 23:32 The Shift from Copper to Optical Connections 26:13 Challenges and Reliability of Lasers 30:47 Understanding Co-Packaged Optics 34:17 Market Dynamics: Demand and Supply of Lasers 40:46 Emerging Technologies: Optical Circuit Switches

The optical networking supercycle is here! In this podcast, and Vikram Sekar go through all the tech jargon and explain what everything means. In just about 45 mins, you will know everything required to keep up with the next revolution in AI. Chapters 00:00 Introduction to AI and CPU Bottlenecks 03:00 The Rise of Silicon Photonics 06:01 Understanding Optical Networking and Data Centers 08:49 Scale Across: Connecting Data Centers 11:56 Scale Out: Optimizing Data Center Connectivity 14:53 Scale Up: The Future of GPU Connectivity 23:32 The Shift from Copper to Optical Connections 26:13 Challenges and Reliability of Lasers 30:47 Understanding Co-Packaged Optics 34:17 Market Dynamics: Demand and Supply of Lasers 40:46 Emerging Technologies: Optical Circuit Switches

Semi Doped

21,842 Aufrufe • vor 6 Monaten

THE 5 BIGGEST BOTTLENECKS POWERING THE AI ECONOMY The way I'm thinking about AI winners today is that the market is moving beyond the simple question of which mega-cap company spends most on AI and has been rewarding the companies that control the scarce inputs, contracted capacity, data movement, power infrastructure, edge compute and workflow layers that make the AI economy function. These are the five bottlenecks I'm watching most: • Memory | $MU, Samsung, SK Hynix Memory is the clearest scarce input because HBM feeds the accelerator, only a few companies can make it at volume and buyers are locking in supply through long-term agreements that create revenue visibility through the end of the decade. • Connectivity | $AVGO, $MRVL, $ALAB, $CRDO, $AAOI, $ANET Connectivity determines whether AI clusters can move data fast enough because training runs span tens of thousands of chips that need to act like one machine. Once copper runs out of reach that causes the cluster depends on optics, retimers, switches and custom silicon to keep the system moving. • Power | $CEG, $VST, $GEV, $FPS, $VRT, $NVTS, $TLN, $ON Power determines whether new AI data centers can actually come online because the binding constraint is shifting from getting chips to getting megawatts so the value flows to the companies that control generation, grid equipment, power delivery, thermal management and efficiency. • Compute | $NBIS, $CIFR, $IREN, $APLD, $WULF, $CORZ, $CRWV Compute capacity is overflow layer when hyperscalers are sold out. Capital alone doesn't guarantee GPU access which is why buyers are signing multi-year contracts for clusters before they are even fully built. • CPU | $NVDA, $AMD, $INTC, $ARM, $QCOM On-device CPU (edge compute) becomes next bottleneck as AI moves into inference, agents, PCs, phones, vehicles and physical devices.

THE 5 BIGGEST BOTTLENECKS POWERING THE AI ECONOMY The way I'm thinking about AI winners today is that the market is moving beyond the simple question of which mega-cap company spends most on AI and has been rewarding the companies that control the scarce inputs, contracted capacity, data movement, power infrastructure, edge compute and workflow layers that make the AI economy function. These are the five bottlenecks I'm watching most: • Memory | $MU, Samsung, SK Hynix Memory is the clearest scarce input because HBM feeds the accelerator, only a few companies can make it at volume and buyers are locking in supply through long-term agreements that create revenue visibility through the end of the decade. • Connectivity | $AVGO, $MRVL, $ALAB, $CRDO, $AAOI, $ANET Connectivity determines whether AI clusters can move data fast enough because training runs span tens of thousands of chips that need to act like one machine. Once copper runs out of reach that causes the cluster depends on optics, retimers, switches and custom silicon to keep the system moving. • Power | $CEG, $VST, $GEV, $FPS, $VRT, $NVTS, $TLN, $ON Power determines whether new AI data centers can actually come online because the binding constraint is shifting from getting chips to getting megawatts so the value flows to the companies that control generation, grid equipment, power delivery, thermal management and efficiency. • Compute | $NBIS, $CIFR, $IREN, $APLD, $WULF, $CORZ, $CRWV Compute capacity is overflow layer when hyperscalers are sold out. Capital alone doesn't guarantee GPU access which is why buyers are signing multi-year contracts for clusters before they are even fully built. • CPU | $NVDA, $AMD, $INTC, $ARM, $QCOM On-device CPU (edge compute) becomes next bottleneck as AI moves into inference, agents, PCs, phones, vehicles and physical devices.

Shay Boloor

109,467 Aufrufe • vor 2 Monaten

$MU $SNDK $LITE $VRT NVIDIA and Groq: 2nd and 3rd Order Strategic Infrastructure Effects and Market Implications Public reporting indicates NVIDIA has agreed to acquire Groq for approximately $20,000,000,000 in cash, while excluding Groq’s nascent cloud business from the transaction perimeter. The reported carve-out materially constrains the immediate, direct linkage from the acquisition to incremental, NVIDIA-controlled data center capacity build-out because GroqCloud appears to be the principal channel through which Groq hardware is currently monetized at scale as a service. The infrastructure-market implications therefore depend primarily on post-close product strategy: whether NVIDIA (1) commercializes Groq silicon as a distinct inference product line and drives broad deployment through OEM/ODM channels and partners, (2) uses the acquisition mainly to absorb IP and talent while de-emphasizing standalone Groq hardware volumes, or (3) uses Groq technology to reshape NVIDIA’s own inference systems and networking roadmaps. The dominant transmission mechanism into memory, networking, and facility infrastructure markets is the degree to which NVIDIA shifts incremental inference deployments away from GPU architectures that are tightly coupled to external high-bandwidth memory (HBM) and toward Groq’s current architecture, which emphasizes large on-chip SRAM, deterministic compiler-scheduled execution, and direct chip-to-chip connectivity. Independent and company-published materials describe Groq’s current-generation approach as having no external memory, keeping weights and KV cache on-chip during processing, and requiring model sharding across multiple chips due to limited on-chip SRAM per device. That architectural choice is directionally HBM-negative on a per-accelerator basis and ambiguous for DRAM, NAND, networking, power, and cooling on a per-token basis because the design can reduce memory wall losses and tail-latency overhead while potentially increasing the number of chips and interconnect endpoints required to serve large models and long-context workloads. HBM implications are the most mechanically straightforward but should be framed as second-derivative rather than absolute. If Groq-class inference silicon meaningfully displaces NVIDIA GPU-based inference deployments, incremental HBM bit demand tied to inference growth could be reduced relative to a GPU-only baseline because Groq’s current approach does not appear to attach HBM stacks to each accelerator. However, current market structure suggests HBM remains supply-constrained and is being pulled by multiple vectors including continued GPU training scale and high-capacity inference configurations, with leading suppliers signaling tight conditions extending beyond 2026. In that environment, reduced inference-driven HBM intensity could primarily reallocate scarce HBM supply toward higher-end training and premium inference GPUs rather than creating an outright volume collapse, preserving high utilization of HBM capacity while potentially affecting the slope of pricing power and capacity expansion urgency over a multi-year horizon. The key downside scenario for the HBM complex would be a durable architectural bifurcation where “good-enough” inference shifts disproportionately to HBM-less ASICs across a broad swath of deployments (latency-sensitive, batch-1, cost-per-token optimized), while training remains GPU-HBM dominated; such a split would reduce the portion of future inference compute that naturally monetizes through HBM content and could compress the incremental HBM-per-AI-dollar ratio. The key upside/neutral scenario for HBM is that the supply chain remains fully allocated regardless, with NVIDIA using any “freed” HBM to ship more high-end GPUs into training and long-context inference, especially as roadmaps increase HBM per GPU, sustaining robust aggregate bit demand even if inference becomes more heterogeneous. Conventional DRAM implications split into 2 channels: (1) DRAM wafer capacity diversion into HBM and (2) DDR content per server in AI clusters. Supplier commentary indicates that AI-driven memory demand is supporting elevated DRAM markets more broadly, and HBM production is resource-intensive versus conventional DRAM, tightening supply for DDR products in parallel. A meaningful NVIDIA pivot to an inference architecture that reduces HBM dependence could, at the margin, ease the most acute HBM-driven bottlenecks and allow memory manufacturers more flexibility in balancing DRAM mix, which could be modestly DDR-positive on the supply side (less crowding-out) even if it is DDR-neutral or slightly negative on the demand side (if per-node CPU/DDR requirements decline due to more efficient accelerator utilization). The dominant practical outcome is likely that DDR demand remains supported by broad AI server proliferation and increasing memory footprints at the system level (CPUs, networking stacks, caching layers, retrieval-augmented pipelines), while HBM remains the premium profit pool; therefore, any HBM displacement that increases total server volumes could indirectly keep DDR demand resilient even if DDR per accelerator is not rising materially. NAND flash implications are comparatively indirect and volume-driven rather than architecture-driven. Inference clusters require SSD capacity for model storage, container images, logging, and increasingly for fast local retrieval indices and embedding stores, but the storage footprint per unit of compute is typically smaller than in training pipelines that stage large datasets and checkpoints. If NVIDIA uses Groq to lower inference cost and latency enough to expand the total number of inference deployment locations (regional colocation, enterprise on-prem, sovereign footprints), aggregate SSD attach could rise through geographic fragmentation and replication of model artifacts across more sites, even if per-site storage is modest. The NAND effect is therefore likely to be demand-broadening and mix-positive (datacenter SSDs) but not a primary swing factor versus the macro AI capex cycle and consumer/device cycles. Hard disk drive (HDD) markets should see negligible direct sensitivity because nearline HDD demand is driven by bulk storage and cloud archiving economics, while inference acceleration choices primarily reshape compute and network layers; any HDD benefit would be a tertiary function of overall data center square footage expansion rather than a direct consequence of Groq silicon displacing GPUs. Optical networking implications require separating (1) intra-cluster back-end fabrics that connect accelerators and (2) front-end / data center interconnect (DCI) that connects sites and regions. Groq’s own positioning and third-party reporting suggest scaling beyond a single node or rack relies on high-bandwidth fabrics and, in some described configurations, optical interconnect scaling across hundreds of chips. If NVIDIA commercializes Groq at scale, 2 offsetting forces emerge: lower cost-per-token and improved latency could expand inference throughput and drive more east-west traffic, increasing demand for high-speed switching and optics; conversely, if Groq delivers materially higher utilization and tokens per unit of network bandwidth for certain workloads, the network required per served token could decline. Public NVIDIA materials already indicate an aggressive photonics roadmap aimed at scaling AI factories, including co-packaged optics (CPO) switches and explicit collaboration with Coherent and Lumentum in the silicon photonics supply chain. That linkage is important because it suggests that, independent of Groq, NVIDIA is already pushing optics integration deeper into the switch package to reduce power and increase resiliency; Groq increases the strategic incentive to reduce network power and latency if inference becomes even more distributed and latency-sensitive. For Lumentum and Coherent specifically, the net implication is less about “more optics versus fewer optics” and more about a shift in optics form factor and value capture. Co-packaged optics can reduce reliance on pluggable transceivers in some switch architectures while increasing demand for integrated photonic engines, lasers, fiber attach, packaging processes, and component-level supply. NVIDIA’s own announcements explicitly position Coherent and Lumentum as collaborators in creating the integrated silicon/optics process and supply chain for photonics switches. If Groq accelerates the transition to very large-scale fabrics (more endpoints, higher port speeds, tighter power envelopes), that tends to pull forward CPO adoption and amplifies demand for the underlying photonics components even if the conventional pluggable module TAM is structurally pressured over time. If Groq instead pushes inference toward smaller, more localized pods (closer to users, more regional colocation), that can be optics-positive for DCI and metro connectivity because more sites must be interconnected at high bandwidth with low latency, favoring coherent optics and high-speed interconnect between facilities. The principal risk for optics suppliers is timing and margin structure: a faster move to NVIDIA-driven integrated photonics could concentrate bargaining power and compress margins for commoditized transceiver modules while favoring suppliers with differentiated lasers, integration capability, and qualification depth in NVIDIA’s CPO ecosystem. AEC and copper interconnect implications hinge on whether Groq deployment increases the density of short-reach links inside racks and rows. High-speed copper remains structurally advantaged at very short distances on cost, power, and serviceability, but reaches become constrained as lane speeds and aggregate bandwidth rise, creating a role for active electrical cables (AECs), retimers, and signal-conditioning silicon. Credo explicitly positions its AEC products as enabling reliable lossless 800G connectivity for AI clusters, and the company has highlighted participation at NVIDIA GTC with content focused on extending PCIe/CXL using AECs, indicating relevance to next-generation system topologies that require longer reach and higher signal integrity than passive copper can deliver. If NVIDIA turns Groq into a widely deployed inference card or chassis product, the likely near-term effect is AEC-positive because (1) more inference throughput tends to increase top-of-rack connectivity requirements, (2) distributing inference across more racks and sites increases short-reach links per unit of delivered service, and (3) PCIe-attached accelerator architectures tend to require robust signal conditioning as systems move to PCIe 6.x and beyond. Groq workshop materials explicitly reference GroqCard and GroqNode form factors, reinforcing that PCIe-attached deployment has been central to Groq’s current packaging strategy. The main countervailing risk is that Groq’s deterministic chip-to-chip fabric could be implemented primarily through backplanes and direct board-level connectivity that reduces the need for merchant AECs inside the box; in that case, incremental AEC demand would concentrate more in rack-to-switch and node-to-fabric links rather than within-chassis chip fabrics. Astera Labs implications are connectivity-architecture sensitive and, on balance, skew positive if NVIDIA increases heterogeneity and disaggregation in AI systems. NVIDIA has publicly positioned NVLink Fusion as a pathway for partners to build semi-custom AI infrastructure and has explicitly identified Astera Labs as a partner in that ecosystem, with Astera describing NVLink-related solutions expanding its connectivity platform across PCIe, CXL, and Ethernet plus fleet observability software. A Groq acquisition increases the probability that NVIDIA offers a broader menu of accelerators (training GPUs, inference-focused ASICs) and therefore increases the importance of scalable, high-reliability connectivity, retiming, switching, and telemetry across mixed topologies. If Groq silicon remains PCIe-attached in many deployments, PCIe 6.x retimers/switches and active cable modules become more central, aligning with Astera’s core portfolio. If NVIDIA instead integrates Groq concepts into scale-up fabrics (NVLink-like domains) or uses Groq to expand into inference “appliances” that must be rapidly deployed in colocation environments, the need for standard-compliant, serviceable connectivity with strong RAS/telemetry increases, again aligning with Astera’s positioning. Power equipment and cooling implications for Vertiv and adjacent suppliers should be viewed through the lens of rack power density, cooling modality (air vs liquid), and site deployment model (hyperscale campuses vs distributed colocation/enterprise). Groq claims its LPU and rack designs are “air-cooled by design” and require no complex cooling and power infrastructure, and third-party reporting has described Groq’s approach as relying on parallelism across many lower-power units rather than extreme per-chip performance. If NVIDIA scales Groq as a mainstream inference platform, the mix of data center cooling spend could shift modestly away from the highest-density liquid-cooled racks toward more air-cooled or hybrid deployments, particularly for inference pods placed in existing facilities that cannot easily retrofit for very high rack heat flux. That would be a mix headwind for suppliers most levered exclusively to high-end liquid cooling attachments per rack, but it is not necessarily a volume headwind for Vertiv given the company’s broad exposure to both power and cooling infrastructure and the likelihood that total AI deployment locations expand. Vertiv’s own industry commentary emphasizes that AI racks require higher power-density UPS, batteries, power distribution equipment, and switchgear capable of handling rapid load transients, and that hybrid cooling systems will evolve across deployment environments. Those statements align with a world where inference growth increases the count of powered racks and raises the operational complexity of power delivery even if per-rack density is lower than the most extreme training clusters. The most material infrastructure impact may occur outside the rack and upstream of the data hall: grid interconnects, substations, transformers, switchgear, generators, and utility-scale generation additions. Recent regulatory actions in the U.S. highlight that projected data center demand is already driving large planned increases in electricity generation capacity, underscoring that power availability is a binding constraint. In that context, an inference architecture that lowers joules per token could reduce the power required per unit of inference delivered, but it can also accelerate demand by lowering cost and improving latency, increasing the total volume of inference served (a classic rebound effect). The net outcome is likely continued, elevated demand for power infrastructure even if efficiency improves, with the key swing factor being whether AI capex remains on a multi-year growth trajectory or enters a digestion phase. Other data center infrastructure implications include server/ODM mix, facility design standardization, and networking architecture choices. If NVIDIA positions Groq-based inference as a broadly distributable “standard server + accelerator” solution rather than as an integrated, liquid-cooled rack like GB200 NVL72, spend could shift toward more conventional air-cooled server designs, higher unit volumes of mainstream racks, and faster deployment in colocation footprints, increasing demand for modular power rooms, busways, and rapidly deployable cooling solutions. If NVIDIA instead integrates Groq into its “AI factory” paradigm, the primary effect is likely acceleration of dense back-end fabric build-outs and a faster push toward photonics switching, increasing demand for fiber plant, connectors, and integrated optics supply chains while potentially compressing the lifecycle of transitional architectures based on pluggable optics and mid-reach copper. NVIDIA’s stated roadmap toward co-packaged optics and silicon photonics switches is already oriented toward scaling to very large GPU counts; adding a high-end inference ASIC increases the strategic importance of power-efficient, low-latency fabrics because inference economics become increasingly sensitive to network overhead as compute cost declines. Across the covered segments, the most defensible base case is limited near-term dislocation and a medium-term increase in uncertainty around memory intensity per unit of inference growth. HBM faces the clearest relative risk from an HBM-less inference platform, but supply tightness and GPU training roadmaps reduce the probability of an absolute demand shock over the next 12–24 months. Optical, AEC/copper, and power/cooling are more likely to remain volume-supported because they scale with endpoint count, deployment fragmentation, and total data center footprint, and those tend to rise when inference becomes cheaper and more widely deployed. The highest-conviction second-order effect is a shift in infrastructure mix: incrementally more distributed inference deployments (favoring colocation power/cooling standardization, DCI optics, and serviceable short-reach interconnect) and a gradual migration from pluggable optics toward integrated photonics in back-end fabrics (favoring suppliers positioned in the CPO ecosystem).

$MU $SNDK $LITE $VRT NVIDIA and Groq: 2nd and 3rd Order Strategic Infrastructure Effects and Market Implications Public reporting indicates NVIDIA has agreed to acquire Groq for approximately $20,000,000,000 in cash, while excluding Groq’s nascent cloud business from the transaction perimeter. The reported carve-out materially constrains the immediate, direct linkage from the acquisition to incremental, NVIDIA-controlled data center capacity build-out because GroqCloud appears to be the principal channel through which Groq hardware is currently monetized at scale as a service. The infrastructure-market implications therefore depend primarily on post-close product strategy: whether NVIDIA (1) commercializes Groq silicon as a distinct inference product line and drives broad deployment through OEM/ODM channels and partners, (2) uses the acquisition mainly to absorb IP and talent while de-emphasizing standalone Groq hardware volumes, or (3) uses Groq technology to reshape NVIDIA’s own inference systems and networking roadmaps. The dominant transmission mechanism into memory, networking, and facility infrastructure markets is the degree to which NVIDIA shifts incremental inference deployments away from GPU architectures that are tightly coupled to external high-bandwidth memory (HBM) and toward Groq’s current architecture, which emphasizes large on-chip SRAM, deterministic compiler-scheduled execution, and direct chip-to-chip connectivity. Independent and company-published materials describe Groq’s current-generation approach as having no external memory, keeping weights and KV cache on-chip during processing, and requiring model sharding across multiple chips due to limited on-chip SRAM per device. That architectural choice is directionally HBM-negative on a per-accelerator basis and ambiguous for DRAM, NAND, networking, power, and cooling on a per-token basis because the design can reduce memory wall losses and tail-latency overhead while potentially increasing the number of chips and interconnect endpoints required to serve large models and long-context workloads. HBM implications are the most mechanically straightforward but should be framed as second-derivative rather than absolute. If Groq-class inference silicon meaningfully displaces NVIDIA GPU-based inference deployments, incremental HBM bit demand tied to inference growth could be reduced relative to a GPU-only baseline because Groq’s current approach does not appear to attach HBM stacks to each accelerator. However, current market structure suggests HBM remains supply-constrained and is being pulled by multiple vectors including continued GPU training scale and high-capacity inference configurations, with leading suppliers signaling tight conditions extending beyond 2026. In that environment, reduced inference-driven HBM intensity could primarily reallocate scarce HBM supply toward higher-end training and premium inference GPUs rather than creating an outright volume collapse, preserving high utilization of HBM capacity while potentially affecting the slope of pricing power and capacity expansion urgency over a multi-year horizon. The key downside scenario for the HBM complex would be a durable architectural bifurcation where “good-enough” inference shifts disproportionately to HBM-less ASICs across a broad swath of deployments (latency-sensitive, batch-1, cost-per-token optimized), while training remains GPU-HBM dominated; such a split would reduce the portion of future inference compute that naturally monetizes through HBM content and could compress the incremental HBM-per-AI-dollar ratio. The key upside/neutral scenario for HBM is that the supply chain remains fully allocated regardless, with NVIDIA using any “freed” HBM to ship more high-end GPUs into training and long-context inference, especially as roadmaps increase HBM per GPU, sustaining robust aggregate bit demand even if inference becomes more heterogeneous. Conventional DRAM implications split into 2 channels: (1) DRAM wafer capacity diversion into HBM and (2) DDR content per server in AI clusters. Supplier commentary indicates that AI-driven memory demand is supporting elevated DRAM markets more broadly, and HBM production is resource-intensive versus conventional DRAM, tightening supply for DDR products in parallel. A meaningful NVIDIA pivot to an inference architecture that reduces HBM dependence could, at the margin, ease the most acute HBM-driven bottlenecks and allow memory manufacturers more flexibility in balancing DRAM mix, which could be modestly DDR-positive on the supply side (less crowding-out) even if it is DDR-neutral or slightly negative on the demand side (if per-node CPU/DDR requirements decline due to more efficient accelerator utilization). The dominant practical outcome is likely that DDR demand remains supported by broad AI server proliferation and increasing memory footprints at the system level (CPUs, networking stacks, caching layers, retrieval-augmented pipelines), while HBM remains the premium profit pool; therefore, any HBM displacement that increases total server volumes could indirectly keep DDR demand resilient even if DDR per accelerator is not rising materially. NAND flash implications are comparatively indirect and volume-driven rather than architecture-driven. Inference clusters require SSD capacity for model storage, container images, logging, and increasingly for fast local retrieval indices and embedding stores, but the storage footprint per unit of compute is typically smaller than in training pipelines that stage large datasets and checkpoints. If NVIDIA uses Groq to lower inference cost and latency enough to expand the total number of inference deployment locations (regional colocation, enterprise on-prem, sovereign footprints), aggregate SSD attach could rise through geographic fragmentation and replication of model artifacts across more sites, even if per-site storage is modest. The NAND effect is therefore likely to be demand-broadening and mix-positive (datacenter SSDs) but not a primary swing factor versus the macro AI capex cycle and consumer/device cycles. Hard disk drive (HDD) markets should see negligible direct sensitivity because nearline HDD demand is driven by bulk storage and cloud archiving economics, while inference acceleration choices primarily reshape compute and network layers; any HDD benefit would be a tertiary function of overall data center square footage expansion rather than a direct consequence of Groq silicon displacing GPUs. Optical networking implications require separating (1) intra-cluster back-end fabrics that connect accelerators and (2) front-end / data center interconnect (DCI) that connects sites and regions. Groq’s own positioning and third-party reporting suggest scaling beyond a single node or rack relies on high-bandwidth fabrics and, in some described configurations, optical interconnect scaling across hundreds of chips. If NVIDIA commercializes Groq at scale, 2 offsetting forces emerge: lower cost-per-token and improved latency could expand inference throughput and drive more east-west traffic, increasing demand for high-speed switching and optics; conversely, if Groq delivers materially higher utilization and tokens per unit of network bandwidth for certain workloads, the network required per served token could decline. Public NVIDIA materials already indicate an aggressive photonics roadmap aimed at scaling AI factories, including co-packaged optics (CPO) switches and explicit collaboration with Coherent and Lumentum in the silicon photonics supply chain. That linkage is important because it suggests that, independent of Groq, NVIDIA is already pushing optics integration deeper into the switch package to reduce power and increase resiliency; Groq increases the strategic incentive to reduce network power and latency if inference becomes even more distributed and latency-sensitive. For Lumentum and Coherent specifically, the net implication is less about “more optics versus fewer optics” and more about a shift in optics form factor and value capture. Co-packaged optics can reduce reliance on pluggable transceivers in some switch architectures while increasing demand for integrated photonic engines, lasers, fiber attach, packaging processes, and component-level supply. NVIDIA’s own announcements explicitly position Coherent and Lumentum as collaborators in creating the integrated silicon/optics process and supply chain for photonics switches. If Groq accelerates the transition to very large-scale fabrics (more endpoints, higher port speeds, tighter power envelopes), that tends to pull forward CPO adoption and amplifies demand for the underlying photonics components even if the conventional pluggable module TAM is structurally pressured over time. If Groq instead pushes inference toward smaller, more localized pods (closer to users, more regional colocation), that can be optics-positive for DCI and metro connectivity because more sites must be interconnected at high bandwidth with low latency, favoring coherent optics and high-speed interconnect between facilities. The principal risk for optics suppliers is timing and margin structure: a faster move to NVIDIA-driven integrated photonics could concentrate bargaining power and compress margins for commoditized transceiver modules while favoring suppliers with differentiated lasers, integration capability, and qualification depth in NVIDIA’s CPO ecosystem. AEC and copper interconnect implications hinge on whether Groq deployment increases the density of short-reach links inside racks and rows. High-speed copper remains structurally advantaged at very short distances on cost, power, and serviceability, but reaches become constrained as lane speeds and aggregate bandwidth rise, creating a role for active electrical cables (AECs), retimers, and signal-conditioning silicon. Credo explicitly positions its AEC products as enabling reliable lossless 800G connectivity for AI clusters, and the company has highlighted participation at NVIDIA GTC with content focused on extending PCIe/CXL using AECs, indicating relevance to next-generation system topologies that require longer reach and higher signal integrity than passive copper can deliver. If NVIDIA turns Groq into a widely deployed inference card or chassis product, the likely near-term effect is AEC-positive because (1) more inference throughput tends to increase top-of-rack connectivity requirements, (2) distributing inference across more racks and sites increases short-reach links per unit of delivered service, and (3) PCIe-attached accelerator architectures tend to require robust signal conditioning as systems move to PCIe 6.x and beyond. Groq workshop materials explicitly reference GroqCard and GroqNode form factors, reinforcing that PCIe-attached deployment has been central to Groq’s current packaging strategy. The main countervailing risk is that Groq’s deterministic chip-to-chip fabric could be implemented primarily through backplanes and direct board-level connectivity that reduces the need for merchant AECs inside the box; in that case, incremental AEC demand would concentrate more in rack-to-switch and node-to-fabric links rather than within-chassis chip fabrics. Astera Labs implications are connectivity-architecture sensitive and, on balance, skew positive if NVIDIA increases heterogeneity and disaggregation in AI systems. NVIDIA has publicly positioned NVLink Fusion as a pathway for partners to build semi-custom AI infrastructure and has explicitly identified Astera Labs as a partner in that ecosystem, with Astera describing NVLink-related solutions expanding its connectivity platform across PCIe, CXL, and Ethernet plus fleet observability software. A Groq acquisition increases the probability that NVIDIA offers a broader menu of accelerators (training GPUs, inference-focused ASICs) and therefore increases the importance of scalable, high-reliability connectivity, retiming, switching, and telemetry across mixed topologies. If Groq silicon remains PCIe-attached in many deployments, PCIe 6.x retimers/switches and active cable modules become more central, aligning with Astera’s core portfolio. If NVIDIA instead integrates Groq concepts into scale-up fabrics (NVLink-like domains) or uses Groq to expand into inference “appliances” that must be rapidly deployed in colocation environments, the need for standard-compliant, serviceable connectivity with strong RAS/telemetry increases, again aligning with Astera’s positioning. Power equipment and cooling implications for Vertiv and adjacent suppliers should be viewed through the lens of rack power density, cooling modality (air vs liquid), and site deployment model (hyperscale campuses vs distributed colocation/enterprise). Groq claims its LPU and rack designs are “air-cooled by design” and require no complex cooling and power infrastructure, and third-party reporting has described Groq’s approach as relying on parallelism across many lower-power units rather than extreme per-chip performance. If NVIDIA scales Groq as a mainstream inference platform, the mix of data center cooling spend could shift modestly away from the highest-density liquid-cooled racks toward more air-cooled or hybrid deployments, particularly for inference pods placed in existing facilities that cannot easily retrofit for very high rack heat flux. That would be a mix headwind for suppliers most levered exclusively to high-end liquid cooling attachments per rack, but it is not necessarily a volume headwind for Vertiv given the company’s broad exposure to both power and cooling infrastructure and the likelihood that total AI deployment locations expand. Vertiv’s own industry commentary emphasizes that AI racks require higher power-density UPS, batteries, power distribution equipment, and switchgear capable of handling rapid load transients, and that hybrid cooling systems will evolve across deployment environments. Those statements align with a world where inference growth increases the count of powered racks and raises the operational complexity of power delivery even if per-rack density is lower than the most extreme training clusters. The most material infrastructure impact may occur outside the rack and upstream of the data hall: grid interconnects, substations, transformers, switchgear, generators, and utility-scale generation additions. Recent regulatory actions in the U.S. highlight that projected data center demand is already driving large planned increases in electricity generation capacity, underscoring that power availability is a binding constraint. In that context, an inference architecture that lowers joules per token could reduce the power required per unit of inference delivered, but it can also accelerate demand by lowering cost and improving latency, increasing the total volume of inference served (a classic rebound effect). The net outcome is likely continued, elevated demand for power infrastructure even if efficiency improves, with the key swing factor being whether AI capex remains on a multi-year growth trajectory or enters a digestion phase. Other data center infrastructure implications include server/ODM mix, facility design standardization, and networking architecture choices. If NVIDIA positions Groq-based inference as a broadly distributable “standard server + accelerator” solution rather than as an integrated, liquid-cooled rack like GB200 NVL72, spend could shift toward more conventional air-cooled server designs, higher unit volumes of mainstream racks, and faster deployment in colocation footprints, increasing demand for modular power rooms, busways, and rapidly deployable cooling solutions. If NVIDIA instead integrates Groq into its “AI factory” paradigm, the primary effect is likely acceleration of dense back-end fabric build-outs and a faster push toward photonics switching, increasing demand for fiber plant, connectors, and integrated optics supply chains while potentially compressing the lifecycle of transitional architectures based on pluggable optics and mid-reach copper. NVIDIA’s stated roadmap toward co-packaged optics and silicon photonics switches is already oriented toward scaling to very large GPU counts; adding a high-end inference ASIC increases the strategic importance of power-efficient, low-latency fabrics because inference economics become increasingly sensitive to network overhead as compute cost declines. Across the covered segments, the most defensible base case is limited near-term dislocation and a medium-term increase in uncertainty around memory intensity per unit of inference growth. HBM faces the clearest relative risk from an HBM-less inference platform, but supply tightness and GPU training roadmaps reduce the probability of an absolute demand shock over the next 12–24 months. Optical, AEC/copper, and power/cooling are more likely to remain volume-supported because they scale with endpoint count, deployment fragmentation, and total data center footprint, and those tend to rise when inference becomes cheaper and more widely deployed. The highest-conviction second-order effect is a shift in infrastructure mix: incrementally more distributed inference deployments (favoring colocation power/cooling standardization, DCI optics, and serviceable short-reach interconnect) and a gradual migration from pluggable optics toward integrated photonics in back-end fabrics (favoring suppliers positioned in the CPO ecosystem).

TheValueist

76,179 Aufrufe • vor 8 Monaten

Jensen Huang is investing in every photonics company he can find and the reason why tells you everything about where AI is headed (Save this). Lip-Bu Tan, the CEO of Intel says, when he looks for investment opportunities, he looks for the bottleneck and right now, the bottleneck is the interconnect, the pipes that move data between chips inside an AI data center. That is why he backed Credo Semiconductor, Astera Labs and Celestial AI on the optical side. Here is the simple version of what the interconnect bottleneck actually means. Think of an AI data center like a city, the GPUs are the buildings where all the work happens but for those buildings to function, you need roads connecting them, fast roads that can carry enormous traffic without congestion. And those roads are now the single biggest constraint on AI performance. As clusters scale to hundreds of thousands of GPUs, traditional copper wiring is hitting its physical limits and that is where this entire sector comes in. Credo Semiconductor (CRDO) is the most direct pure play on this theme, Credo makes high speed cables and optical chips that connect GPUs inside data center racks. Their revenue tripled in fiscal 2026 to $1.3 billion, growing 272% year over year at its peak and four of the world's largest hyperscalers each individually account for more than 10% of Credo's revenue. Astera Labs (ALAB) solves the connection problem between different chip types. Astera makes the PCIe and connectivity chips that manage data flow between GPUs, CPUs, and memory without errors or slowdowns. Their revenue grew 93% year over year to $308 million in Q1 2026 alone. The optical companies are where the longer-term and potentially larger opportunity lives. Copper has physical limits, you can only push electrical signals so far before the signal degrades, the heat spikes and power consumption explodes. The solution is light, fiber optic connections that move data using photons instead of electrons which is faster, cooler and far more energy efficient. Jensen Huang made this clear at Computex 2026 because copper works as long as physically possible but at greater distances and larger scale, optics takes over. Coherent (COHR) is the most established optical company in this space. Coherent makes the lasers, transceivers, and optical components at the foundation of all fiber optic communications. Nvidia signed a multibillion-dollar purchase commitment and invested $2 billion directly into the company and their customer order books are already extending out to 2028. Marvell (MRVL) is the most comprehensive bet across the entire connectivity stack. Marvell makes chips for optical networking, PCIe switching and custom AI silicon. Jensen Huang called Marvell the next trillion dollar company at Computex 2026 and backed it with a $2 billion Nvidia investment. Marvell also acquired Celestial AI, the exact company Lip-Bu Tan backed for $3.25 billion, gaining photonic fabric technology delivering 16 terabits per second of bandwidth. Lumentum (LITE), Corning (GLW), and Ciena (CIEN) round out the major public names. Lumentum received a $2 billion Nvidia investment for laser and photonics components. Corning known mostly for phone glass received $500 million from Nvidia for optical connectivity work and is up over 100% year to date. Ciena runs the optical networking systems between data centers and is seeing analyst price targets raised on the back of the AI optics boom. Every time a hyperscaler spends a billion dollars on Nvidia GPUs, the surrounding infrastructure, cables, switches, transceivers, optical components has to be upgraded to match. The smarter the GPU gets, the more the interconnect matters. Nvidia has committed at least $6.5 billion to photonics companies in the past 4 months alone and the companies building the roads between the GPUs may end up being just as valuable as the companies building the GPUs themselves. Follow me Melvin for more AI, semis and the next big market themes.

Jensen Huang is investing in every photonics company he can find and the reason why tells you everything about where AI is headed (Save this). Lip-Bu Tan, the CEO of Intel says, when he looks for investment opportunities, he looks for the bottleneck and right now, the bottleneck is the interconnect, the pipes that move data between chips inside an AI data center. That is why he backed Credo Semiconductor, Astera Labs and Celestial AI on the optical side. Here is the simple version of what the interconnect bottleneck actually means. Think of an AI data center like a city, the GPUs are the buildings where all the work happens but for those buildings to function, you need roads connecting them, fast roads that can carry enormous traffic without congestion. And those roads are now the single biggest constraint on AI performance. As clusters scale to hundreds of thousands of GPUs, traditional copper wiring is hitting its physical limits and that is where this entire sector comes in. Credo Semiconductor (CRDO) is the most direct pure play on this theme, Credo makes high speed cables and optical chips that connect GPUs inside data center racks. Their revenue tripled in fiscal 2026 to $1.3 billion, growing 272% year over year at its peak and four of the world's largest hyperscalers each individually account for more than 10% of Credo's revenue. Astera Labs (ALAB) solves the connection problem between different chip types. Astera makes the PCIe and connectivity chips that manage data flow between GPUs, CPUs, and memory without errors or slowdowns. Their revenue grew 93% year over year to $308 million in Q1 2026 alone. The optical companies are where the longer-term and potentially larger opportunity lives. Copper has physical limits, you can only push electrical signals so far before the signal degrades, the heat spikes and power consumption explodes. The solution is light, fiber optic connections that move data using photons instead of electrons which is faster, cooler and far more energy efficient. Jensen Huang made this clear at Computex 2026 because copper works as long as physically possible but at greater distances and larger scale, optics takes over. Coherent (COHR) is the most established optical company in this space. Coherent makes the lasers, transceivers, and optical components at the foundation of all fiber optic communications. Nvidia signed a multibillion-dollar purchase commitment and invested $2 billion directly into the company and their customer order books are already extending out to 2028. Marvell (MRVL) is the most comprehensive bet across the entire connectivity stack. Marvell makes chips for optical networking, PCIe switching and custom AI silicon. Jensen Huang called Marvell the next trillion dollar company at Computex 2026 and backed it with a $2 billion Nvidia investment. Marvell also acquired Celestial AI, the exact company Lip-Bu Tan backed for $3.25 billion, gaining photonic fabric technology delivering 16 terabits per second of bandwidth. Lumentum (LITE), Corning (GLW), and Ciena (CIEN) round out the major public names. Lumentum received a $2 billion Nvidia investment for laser and photonics components. Corning known mostly for phone glass received $500 million from Nvidia for optical connectivity work and is up over 100% year to date. Ciena runs the optical networking systems between data centers and is seeing analyst price targets raised on the back of the AI optics boom. Every time a hyperscaler spends a billion dollars on Nvidia GPUs, the surrounding infrastructure, cables, switches, transceivers, optical components has to be upgraded to match. The smarter the GPU gets, the more the interconnect matters. Nvidia has committed at least $6.5 billion to photonics companies in the past 4 months alone and the companies building the roads between the GPUs may end up being just as valuable as the companies building the GPUs themselves. Follow me Melvin for more AI, semis and the next big market themes.

Melvin

152,406 Aufrufe • vor 2 Monaten

Morgan Stanley just dropped numbers that should make every investor pay attention (Save this). Hyperscalers spent $261B in 2024, they're now projected to spend $1.4T in 2028, a 5x increase in four years and that doesn't even include OpenAI or Anthropic. For the first time, Morgan Stanley is classifying SpaceX as a legitimate hyperscaler alongside Google, Amazon, Microsoft, and Meta. All of it flows into chips, data centers and memory and the supply chain companies sitting beneath the hyperscalers are the ones that will compound quietly. The most underrated play is memory. Micron is the only American HBM supplier, giving it a structural edge in government AI contracts that Samsung and SK Hynix cannot touch. Its entire 2026 HBM4 production is already sold out, revenue nearly tripled to $23.9B, and the memory prices are roughly doubling every year. The cooling problem is one of the most profitable bottlenecks in this entire trade. Vertiv makes the power management, liquid cooling systems and racks that keep GPU clusters from melting and it's up over 100% year to date in 2026 with a $15B+ backlog and guidance raised to $13.5–$14B in full year revenue. Arista Networks (ANET) is the networking infrastructure play, every AI data center needs ultra high speed networking fabric to connect thousands of GPUs together And Arista just doubled its 2026 AI revenue target as the industry shifts from proprietary InfiniBand to Open Ethernet, a shift that plays directly into Arista's strengths. Astera Labs solves the interconnect bottleneck inside data centers, the problem of getting data between chips fast enough to keep up with the GPUs. Revenue grew 93% year over year, it's already profitable, and its customers are Microsoft and Amazon directly. The hyperscalers are the miners and the real money is in the companies selling them the shovels, the electricity, the memory, the cooling, the networking, and the custom silicon. Make sure to follow me Melvin for more underrated infrastructure plays.

Morgan Stanley just dropped numbers that should make every investor pay attention (Save this). Hyperscalers spent $261B in 2024, they're now projected to spend $1.4T in 2028, a 5x increase in four years and that doesn't even include OpenAI or Anthropic. For the first time, Morgan Stanley is classifying SpaceX as a legitimate hyperscaler alongside Google, Amazon, Microsoft, and Meta. All of it flows into chips, data centers and memory and the supply chain companies sitting beneath the hyperscalers are the ones that will compound quietly. The most underrated play is memory. Micron is the only American HBM supplier, giving it a structural edge in government AI contracts that Samsung and SK Hynix cannot touch. Its entire 2026 HBM4 production is already sold out, revenue nearly tripled to $23.9B, and the memory prices are roughly doubling every year. The cooling problem is one of the most profitable bottlenecks in this entire trade. Vertiv makes the power management, liquid cooling systems and racks that keep GPU clusters from melting and it's up over 100% year to date in 2026 with a $15B+ backlog and guidance raised to $13.5–$14B in full year revenue. Arista Networks (ANET) is the networking infrastructure play, every AI data center needs ultra high speed networking fabric to connect thousands of GPUs together And Arista just doubled its 2026 AI revenue target as the industry shifts from proprietary InfiniBand to Open Ethernet, a shift that plays directly into Arista's strengths. Astera Labs solves the interconnect bottleneck inside data centers, the problem of getting data between chips fast enough to keep up with the GPUs. Revenue grew 93% year over year, it's already profitable, and its customers are Microsoft and Amazon directly. The hyperscalers are the miners and the real money is in the companies selling them the shovels, the electricity, the memory, the cooling, the networking, and the custom silicon. Make sure to follow me Melvin for more underrated infrastructure plays.

Melvin

36,068 Aufrufe • vor 1 Monat

AI-scale networking is hitting a new power and density ceiling — and Dell + Broadcom are rethinking the fabric from the thermals up. At SC25, Jim Wynia breaks down why Direct Liquid Cooling (DLC) is becoming a must for next-gen AI networks. Wynia puts it simply: when the hardware runs cooler, everything runs more efficiently — and the AI fabric scales further. Watch the full conversation on how Dell and Broadcom are reclaiming power and performance with the Tomahawk 6 and Enterprise SONiC.

AI-scale networking is hitting a new power and density ceiling — and Dell + Broadcom are rethinking the fabric from the thermals up. At SC25, Jim Wynia breaks down why Direct Liquid Cooling (DLC) is becoming a must for next-gen AI networks. Wynia puts it simply: when the hardware runs cooler, everything runs more efficiently — and the AI fabric scales further. Watch the full conversation on how Dell and Broadcom are reclaiming power and performance with the Tomahawk 6 and Enterprise SONiC.

Six Five Media

66,740 Aufrufe • vor 9 Monaten

Building AI infrastructure at yotta-scale requires more than raw performance; it demands an open, modular rack design that can evolve across product generations, combining leadership compute engines with high-speed networking to connect thousands of accelerators into a single, unified system. Take a look at the Helios. Our rack-scale AI platform powered by MI455X GPUs and EPYC “Venice” CPUs and designed for trillion-parameter training.

Building AI infrastructure at yotta-scale requires more than raw performance; it demands an open, modular rack design that can evolve across product generations, combining leadership compute engines with high-speed networking to connect thousands of accelerators into a single, unified system. Take a look at the Helios. Our rack-scale AI platform powered by MI455X GPUs and EPYC “Venice” CPUs and designed for trillion-parameter training.

AMD

20,832 Aufrufe • vor 8 Monaten

AI workload depends on: power, infrastructure and connectivity. At #NEOM, we're bringing together the conditions that make the next generation of AI compute possible: scalable energy and water capacity, digital connectivity, developed industrial land and infrastructure. Building on this, HUMAIN and DataVoltKSA are advancing the construction of the first AI data center at Oxagon, further strengthening NEOM’s position as a hub for AI infrastructure. Together, we're creating an ecosystem for AI to scale, driving investment, innovation and strategic partnerships that support Saudi Arabia's economic diversification.

AI workload depends on: power, infrastructure and connectivity. At #NEOM, we're bringing together the conditions that make the next generation of AI compute possible: scalable energy and water capacity, digital connectivity, developed industrial land and infrastructure. Building on this, HUMAIN and DataVoltKSA are advancing the construction of the first AI data center at Oxagon, further strengthening NEOM’s position as a hub for AI infrastructure. Together, we're creating an ecosystem for AI to scale, driving investment, innovation and strategic partnerships that support Saudi Arabia's economic diversification.

NEOM

26,414 Aufrufe • vor 6 Tagen

$PLTR is one of the cleanest ways to play the AI cycle because it benefits from the entire infrastructure buildout without financing the data centers, chips, memory, power or networking underneath it. Palantir is also emerging as one of the biggest sovereign AI winners by building the control layer that determines what AI understands, which proprietary data it can access and what decisions or actions it can influence inside an organization. OpenAI and Anthropic create the intelligence while Palantir connects it to permissions, workflows and real operating systems where measurable economic value is produced.

$PLTR is one of the cleanest ways to play the AI cycle because it benefits from the entire infrastructure buildout without financing the data centers, chips, memory, power or networking underneath it. Palantir is also emerging as one of the biggest sovereign AI winners by building the control layer that determines what AI understands, which proprietary data it can access and what decisions or actions it can influence inside an organization. OpenAI and Anthropic create the intelligence while Palantir connects it to permissions, workflows and real operating systems where measurable economic value is produced.

Shay Boloor

200,100 Aufrufe • vor 1 Monat

Deep inside a Norwegian mountain lies one of the most unusual HPC environments in the world: Lefdal Mine DC, where cold seawater siphoned from a nearby fjord cools dense GPU clusters designed for large-scale AI workloads. HRT’s deployment at Lefdal includes DLC racks operating above 125 kW per cabinet, integrated with high-speed interconnects and bespoke storage systems designed to power our cutting-edge quantitative research. In our latest video, we take a look inside the mine at our next-gen HPC and networking infrastructure:

Deep inside a Norwegian mountain lies one of the most unusual HPC environments in the world: Lefdal Mine DC, where cold seawater siphoned from a nearby fjord cools dense GPU clusters designed for large-scale AI workloads. HRT’s deployment at Lefdal includes DLC racks operating above 125 kW per cabinet, integrated with high-speed interconnects and bespoke storage systems designed to power our cutting-edge quantitative research. In our latest video, we take a look inside the mine at our next-gen HPC and networking infrastructure:

Hudson River Trading

160,700 Aufrufe • vor 5 Monaten

A masterclass on Google's TPU v8 Networking. Two TPU chips? Pssh. We already knew workload-specific silicon was here. But two scale-up networking topologies? That's the actual Google TPU news. Workload-specific interconnects. Think about that. New Semi Doped with Vikram Sekar and Austin Lyons. Copper? Yep. Optics? Yep. What we cover: - TPU splits in two: 8t training, 8i inference. - Virgo: 47 Pb/s scale-out fabric, 100% OCS. - Boardfly scale-up: copper PCB + AECs inside racks, OCS between groups. 16 hops → 7. - Training uses 3D torus (Rubik's Cube). - Inference doesn't. Workload-specific topologies now. - Dragonfly traces to a 2008 paper by Kim, Dally, Scott, Abts. Abts went on to build Groq's interconnect before Nvidia. Chapters: 0:00 Intro 0:21 Two TPUs for two workloads 2:31 HBM, SRAM, and Axion CPUs 7:22 Why networking is the new bottleneck 17:14 Virgo: rebuilding scale-out on optics 25:24 3D torus Rubik's Cube scale-up for training 34:50 Boardfly: scale-up for MoE inference 42:07 Workload-specific everything $GOOGL

A masterclass on Google's TPU v8 Networking. Two TPU chips? Pssh. We already knew workload-specific silicon was here. But two scale-up networking topologies? That's the actual Google TPU news. Workload-specific interconnects. Think about that. New Semi Doped with Vikram Sekar and Austin Lyons. Copper? Yep. Optics? Yep. What we cover: - TPU splits in two: 8t training, 8i inference. - Virgo: 47 Pb/s scale-out fabric, 100% OCS. - Boardfly scale-up: copper PCB + AECs inside racks, OCS between groups. 16 hops → 7. - Training uses 3D torus (Rubik's Cube). - Inference doesn't. Workload-specific topologies now. - Dragonfly traces to a 2008 paper by Kim, Dally, Scott, Abts. Abts went on to build Groq's interconnect before Nvidia. Chapters: 0:00 Intro 0:21 Two TPUs for two workloads 2:31 HBM, SRAM, and Axion CPUs 7:22 Why networking is the new bottleneck 17:14 Virgo: rebuilding scale-out on optics 25:24 3D torus Rubik's Cube scale-up for training 34:50 Boardfly: scale-up for MoE inference 42:07 Workload-specific everything $GOOGL

Semi Doped

92,064 Aufrufe • vor 4 Monaten

Elon Musk explains the “series of miracles” xAI had to achieve to bring one gigawatt of power online for its supercomputer cluster in Memphis The xAI team coordinated dozens of gas turbines, faced permit challenges in Tennessee, and ultimately moved the power generation just across the border into Mississippi. From there, they had to run high-voltage transmission lines and build an entire power plant on the Mississippi side and it was very difficult to build that But the deeper point is even more important Most people calculate AI power needs by looking only at the GPUs. That is a serious mistake. A real data center also requires substantial power for networking, CPUs, storage, and most critically cooling In Memphis, where summer temperatures are extreme, peak cooling demand can add roughly 40% more power. On top of that, operators must maintain extra capacity so generators can be taken offline for servicing without shutting down the entire system. This redundancy typically adds another 20 to 25 percent When all of these factors are included, Elon stated that approximately 110,000 GB300 GPUs require roughly 330 megawatts of actual power generation This is why building and powering frontier AI data centers is far more difficult and far more capital intensive than it appears from the outside

Elon Musk explains the “series of miracles” xAI had to achieve to bring one gigawatt of power online for its supercomputer cluster in Memphis The xAI team coordinated dozens of gas turbines, faced permit challenges in Tennessee, and ultimately moved the power generation just across the border into Mississippi. From there, they had to run high-voltage transmission lines and build an entire power plant on the Mississippi side and it was very difficult to build that But the deeper point is even more important Most people calculate AI power needs by looking only at the GPUs. That is a serious mistake. A real data center also requires substantial power for networking, CPUs, storage, and most critically cooling In Memphis, where summer temperatures are extreme, peak cooling demand can add roughly 40% more power. On top of that, operators must maintain extra capacity so generators can be taken offline for servicing without shutting down the entire system. This redundancy typically adds another 20 to 25 percent When all of these factors are included, Elon stated that approximately 110,000 GB300 GPUs require roughly 330 megawatts of actual power generation This is why building and powering frontier AI data centers is far more difficult and far more capital intensive than it appears from the outside

X Freeze

140,377 Aufrufe • vor 4 Monaten

The Week Ahead In Tech in 90 seconds for August 17, 2026: AI infrastructure has three constraints that more accelerators cannot fix: power, software portability and memory. I will be attending ModCon, where Modular, now part of Qualcomm, is scheduled to discuss Mojo 1.0's open road ahead, make additional hardware announcements and present a unified AI compute layer. Six U.S. grid operators and their transmission owners must justify their large-load tariffs to FERC or propose changes. The Hot Chips tutorial program covers HBM, advanced packaging for HBM, a 3D-DRAM accelerator for generative inference and RISC-V. The questions are who pays for grid upgrades, whether Modular stays open across CPU, GPU and NPU, and whether memory can keep pace with AI demand.

The Week Ahead In Tech in 90 seconds for August 17, 2026: AI infrastructure has three constraints that more accelerators cannot fix: power, software portability and memory. I will be attending ModCon, where Modular, now part of Qualcomm, is scheduled to discuss Mojo 1.0's open road ahead, make additional hardware announcements and present a unified AI compute layer. Six U.S. grid operators and their transmission owners must justify their large-load tariffs to FERC or propose changes. The Hot Chips tutorial program covers HBM, advanced packaging for HBM, a 3D-DRAM accelerator for generative inference and RISC-V. The questions are who pays for grid upgrades, whether Modular stays open across CPU, GPU and NPU, and whether memory can keep pace with AI demand.

Patrick Moorhead

15,476 Aufrufe • vor 22 Tagen

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,935 Aufrufe • vor 3 Monaten

This is why Nebius will be a trillion dollar hyperscaler (Save this). Nebius is not building another GPU rental shop but rather building a vertically integrated hyperscaler that owns everything from the physical data center, to the server rack hardware it designs in house, to the software stack, to the inference delivery layer. Nearly every other neocloud is essentially a reseller of someone else's infrastructure but Nebius owns the full stack end to end and that distinction is the entire thesis. Here is why vertical integration is the winning architecture for the inference era. AWS and Azure were architected for general purpose computing and every AI workload they run sits on top of infrastructure that was never designed for it, patched, adapted and optimized after the fact. Nebius was built from day one specifically for AI which means every layer of the stack is purpose built and co optimized. The rack design, the networking topology, the cooling systems and the software that orchestrates it all are engineered together as a single system rather than assembled from parts that were never meant to work together. That architectural difference compounds with every passing quarter as AI workloads grow more complex and the performance gap between purpose built and general purpose infrastructure widens. The software layer is where the real competitive moat lives. Most infrastructure companies think of software as a wrapper around hardware while Nebius thinks of software as the product with hardware as the substrate it controls. The company is building an AI native cloud platform where the software layer handles model serving, inference optimization, fine tuning pipelines and developer tooling as first-class primitives. This matters because inference efficiency is almost entirely a software problem. Two companies running identical GPUs can deliver dramatically different performance and cost per token depending on how intelligently the software schedules, batches and routes inference requests across the cluster. Nebius is also building for a fundamental shift in how AI infrastructure gets consumed. Today, enterprise developers navigate massive cloud service catalogs spinning up clusters, managing configurations and building deep expertise in AWS or GCP-specific tooling. The next generation of builders will simply provision agents to interface with infrastructure directly. Nebius is architecting its software layer for that future , one where the interface between the developer and the compute abstraction layer looks nothing like what AWS built in 2006. The entire available capacity has been sold out every quarter. And that is the best possible validation that what Nebius is building is exactly what the market needs and that the market is willing to commit at a scale that makes the current valuation look like the beginning of a much longer story. Long Nebius and make sure to follow me Melvin for more overlooked AI stocks.

This is why Nebius will be a trillion dollar hyperscaler (Save this). Nebius is not building another GPU rental shop but rather building a vertically integrated hyperscaler that owns everything from the physical data center, to the server rack hardware it designs in house, to the software stack, to the inference delivery layer. Nearly every other neocloud is essentially a reseller of someone else's infrastructure but Nebius owns the full stack end to end and that distinction is the entire thesis. Here is why vertical integration is the winning architecture for the inference era. AWS and Azure were architected for general purpose computing and every AI workload they run sits on top of infrastructure that was never designed for it, patched, adapted and optimized after the fact. Nebius was built from day one specifically for AI which means every layer of the stack is purpose built and co optimized. The rack design, the networking topology, the cooling systems and the software that orchestrates it all are engineered together as a single system rather than assembled from parts that were never meant to work together. That architectural difference compounds with every passing quarter as AI workloads grow more complex and the performance gap between purpose built and general purpose infrastructure widens. The software layer is where the real competitive moat lives. Most infrastructure companies think of software as a wrapper around hardware while Nebius thinks of software as the product with hardware as the substrate it controls. The company is building an AI native cloud platform where the software layer handles model serving, inference optimization, fine tuning pipelines and developer tooling as first-class primitives. This matters because inference efficiency is almost entirely a software problem. Two companies running identical GPUs can deliver dramatically different performance and cost per token depending on how intelligently the software schedules, batches and routes inference requests across the cluster. Nebius is also building for a fundamental shift in how AI infrastructure gets consumed. Today, enterprise developers navigate massive cloud service catalogs spinning up clusters, managing configurations and building deep expertise in AWS or GCP-specific tooling. The next generation of builders will simply provision agents to interface with infrastructure directly. Nebius is architecting its software layer for that future , one where the interface between the developer and the compute abstraction layer looks nothing like what AWS built in 2006. The entire available capacity has been sold out every quarter. And that is the best possible validation that what Nebius is building is exactly what the market needs and that the market is willing to commit at a scale that makes the current valuation look like the beginning of a much longer story. Long Nebius and make sure to follow me Melvin for more overlooked AI stocks.

Melvin

34,306 Aufrufe • vor 2 Monaten

What's the next big trade in AI infrastructure after memory? We asked Meltem Demirors and Kelly Greer: "Memory is ripping now. We think the next narrative is gonna be around networking." "Jensen is doing the exact same thing now that he did with Mellanox in 2019 that turned NVIDIA from a GPU company into a systems and networking company. He's doing it again with Marvell." "Hyperscalers are literally telling you the two biggest bottlenecks for AI spend right now are networking and memory." "The tea leaves are written literally everywhere... the market's just been on the memory track. We're trying to get ahead of that."

What's the next big trade in AI infrastructure after memory? We asked Meltem Demirors and Kelly Greer: "Memory is ripping now. We think the next narrative is gonna be around networking." "Jensen is doing the exact same thing now that he did with Mellanox in 2019 that turned NVIDIA from a GPU company into a systems and networking company. He's doing it again with Marvell." "Hyperscalers are literally telling you the two biggest bottlenecks for AI spend right now are networking and memory." "The tea leaves are written literally everywhere... the market's just been on the memory track. We're trying to get ahead of that."

MTS

35,125 Aufrufe • vor 3 Monaten

DigiPower X CEO michel amar takes you inside the Company’s AI Infrastructure Campus in Columbiana, Alabama, showcasing the technology and infrastructure being deployed for the next generation of high-performance computing and artificial intelligence. The facility features DigiPower X’s ARMS 200 modular infrastructure, designed to support NVIDIA B300 GPUs with advanced direct-to-chip liquid cooling, high-density power delivery, resilient power distribution and modular scalability. Michel highlights the key components behind the deployment, including the IT-load modular system, liquid-cooling infrastructure, switchgear and redundant backup generation designed to support demanding AI workloads. The goal is clear: deploy AI compute faster, more efficiently and at significantly higher density. DigiPower X — Powering Intelligence at Scale. $DGXX $DGX #DigiPowerX #DGXX #AIInfrastructure #DataCenters #ArtificialIntelligence #NVIDIAB300 #NVIDIA #GPUInfrastructure #HighPerformanceComputing #HPC #LiquidCooling #DataCenterInfrastructure #AIDatacenter #DigitalInfrastructure #Alabama

DigiPower X CEO michel amar takes you inside the Company’s AI Infrastructure Campus in Columbiana, Alabama, showcasing the technology and infrastructure being deployed for the next generation of high-performance computing and artificial intelligence. The facility features DigiPower X’s ARMS 200 modular infrastructure, designed to support NVIDIA B300 GPUs with advanced direct-to-chip liquid cooling, high-density power delivery, resilient power distribution and modular scalability. Michel highlights the key components behind the deployment, including the IT-load modular system, liquid-cooling infrastructure, switchgear and redundant backup generation designed to support demanding AI workloads. The goal is clear: deploy AI compute faster, more efficiently and at significantly higher density. DigiPower X — Powering Intelligence at Scale. $DGXX $DGX #DigiPowerX #DGXX #AIInfrastructure #DataCenters #ArtificialIntelligence #NVIDIAB300 #NVIDIA #GPUInfrastructure #HighPerformanceComputing #HPC #LiquidCooling #DataCenterInfrastructure #AIDatacenter #DigitalInfrastructure #Alabama

DigiPower X

73,457 Aufrufe • vor 21 Tagen

$IBM is having its worst day in nearly six decades after its CEO said customers redirected quarterly spending toward servers, storage and memory to secure supply-constrained infrastructure ahead of expected price increases. That comment is huge because it provides one of the clearest real-world examples yet of the AI infrastructure buildout reshaping enterprise technology budgets. Demand for compute, memory, networking, power, cooling and skilled labor is becoming so large that companies are being forced to reallocate capital across the rest of their businesses.

$IBM is having its worst day in nearly six decades after its CEO said customers redirected quarterly spending toward servers, storage and memory to secure supply-constrained infrastructure ahead of expected price increases. That comment is huge because it provides one of the clearest real-world examples yet of the AI infrastructure buildout reshaping enterprise technology budgets. Demand for compute, memory, networking, power, cooling and skilled labor is becoming so large that companies are being forced to reallocate capital across the rest of their businesses.

Shay Boloor

109,549 Aufrufe • vor 1 Monat