Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

@TryTrustAI is building the fastest, cheapest lightweight inference by optimizing KV cache at Y Combinator. Our group of MIT grads and olympiad medalists from Google DeepMind, Jane Street, and NeurIPS/ICML publishers are setting a new standard for cost and latency efficiency.

Hannah Chung

1,867 subscribers

33,169 Aufrufe • vor 1 Monat •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

36 Kommentare

Profilbild von Claire Mao

Claire Maovor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv WOWW so impressive congrats!!

Profilbild von Justin Hong

Justin Hongvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv been waiting for this

Profilbild von Hannah Chung

Hannah Chungvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv me too 🥲

Profilbild von medha

medhavor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup Exciting!

Profilbild von Hannah Chung

Hannah Chungvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup I hope you're hyped

Profilbild von Jeremy

Jeremyvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv So gmi 🚀

Profilbild von Nour Zahzah

Nour Zahzahvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Let’s go

Profilbild von Shaurya Aggarwal

Shaurya Aggarwalvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv cracked team! congrats guys

Profilbild von Hannah Chung

Hannah Chungvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv thank you!!!

Profilbild von aryan mahajan

aryan mahajanvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv 🙌

Profilbild von Hannah Chung

Hannah Chungvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv 🙏

Profilbild von Sumit Singh🇮🇳

Sumit Singh🇮🇳vor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Soo great 🤝👨🏻‍💻🌐 looking to know more . Let's connect

Profilbild von Nikhil Gupta

Nikhil Guptavor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv wait IPhO camp does not the mean medalist

Profilbild von Rick A.F.

Rick A.F.vor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Amazing, cache hits different when its optimised! Lets go, lets improve the efficiency of interference!

Profilbild von Akira

Akiravor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Nice

Profilbild von Ruiyang Chen

Ruiyang Chenvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Curious what’s the use case?

Profilbild von Evan Dickinson

Evan Dickinsonvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Mit!

Profilbild von Luke Gainey

Luke Gaineyvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv FANTASTIC

Profilbild von Dheeraj Kaim

Dheeraj Kaimvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv congrats, hope it works out for you guys

Profilbild von An Engineer's Log

An Engineer's Logvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv what's the largest model you can host, and the price per million token for it ?

Profilbild von Rahul Wagh

Rahul Waghvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv interesting!!

Profilbild von om_lathiya_

om_lathiya_vor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Interesting

Profilbild von Evï Anton

Evï Antonvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv is this any different from

Profilbild von Niraj Adhikari

Niraj Adhikarivor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Looks good!

Profilbild von just a monolith

just a monolithvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv I have nothing to add. But goddamn, you're cute

Profilbild von Marco H

Marco Hvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv So interesting.. i was just running research on this as I just heard KV cache for first time today and algorithm is showing it to me now.

Profilbild von Yoonseok Yang

Yoonseok Yangvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv lets go hannah!!

Profilbild von Santz

Santzvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Congrats on the launch!

Profilbild von John

Johnvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Dope

Profilbild von AkinAyo | UI/UX Designer

AkinAyo | UI/UX Designervor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Impressive

Profilbild von 𝕐𝕖𝕤𝕙𝕦𝕒

𝕐𝕖𝕤𝕙𝕦𝕒vor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Great team 👏🏻💯

Profilbild von E

Evor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Bubble

Profilbild von mr ceo

mr ceovor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Good luck

Profilbild von Chandan kumar

Chandan kumarvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Hii

Profilbild von Lokesh

Lokeshvor 26 Tagen

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Any opening role in product management with visa sponsorship?

Profilbild von Amritesh Anand

Amritesh Anandvor 1 Monat

@TryTrustAI @ycombinator @GoogleDeepMind @JaneStreetGroup @medha_rv Is this something we can try out on our GPU's or you serve endpoints to end clients?

Ähnliche Videos

OneTriangle (YC S26) (YC S26) is building the fastest lightweight, cheap inference and hosting DeepSeek V4 Flash today at $0.15/M input and $0.35/M output tokens. Live at Our group of Massachusetts Institute of Technology (MIT) grads and olympiad medalists from Google DeepMind, Jane Street, and NeurIPS Conference /ICML publishers are setting a new standard for cost and latency efficiency using KV cache optimizations.

OneTriangle (YC S26) (YC S26) is building the fastest lightweight, cheap inference and hosting DeepSeek V4 Flash today at $0.15/M input and $0.35/M output tokens. Live at Our group of Massachusetts Institute of Technology (MIT) grads and olympiad medalists from Google DeepMind, Jane Street, and NeurIPS Conference /ICML publishers are setting a new standard for cost and latency efficiency using KV cache optimizations.

Hannah Chung

94,852 Aufrufe • vor 27 Tagen

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

Reese Chong

53,026 Aufrufe • vor 5 Monaten

KV caching is the fundamental optimization underpinning autoregressive LLM inference. Transformer layers store keys and values from earlier tokens, then reuse them as new tokens are generated. This avoids recomputation, but at the cost of memory capacity and bandwidth.

KV caching is the fundamental optimization underpinning autoregressive LLM inference. Transformer layers store keys and values from earlier tokens, then reuse them as new tokens are generated. This avoids recomputation, but at the cost of memory capacity and bandwidth.

Alec Helbling

36,338 Aufrufe • vor 16 Tagen

$We partnered with Harvey to train a model for Review Table, achieving state-of-the-art accuracy at a fraction of the cost and latency of frontier alternatives. Review Table is one of Harvey’s highest inference volume products. Training an open source model for production traffic required building a representative eval and understanding how real lawyers use the product. Our teams discuss the research process.$

We partnered with Harvey to train a model for Review Table, achieving state-of-the-art accuracy at a fraction of the cost and latency of frontier alternatives. Review Table is one of Harvey’s highest inference volume products. Training an open source model for production traffic required building a representative eval and understanding how real lawyers use the product. Our teams discuss the research process.

Applied Compute

33,883 Aufrufe • vor 1 Monat

AI that responds in under 100ms doesn't happen by accident. Together AI VP of Kernels Dan Fu shares how they use NVIDIA CUDA, TensorRT-LLM, Dynamo and Together ATLAS on NVIDIA Blackwell to power ultra-low-latency inference and long context code generation for Cursor. From building megakernels that fuse an entire model's forward pass into a single launch to enabling real-time voice agents, Together AI is optimizing latency at every layer of the stack.

AI that responds in under 100ms doesn't happen by accident. Together AI VP of Kernels Dan Fu shares how they use NVIDIA CUDA, TensorRT-LLM, Dynamo and Together ATLAS on NVIDIA Blackwell to power ultra-low-latency inference and long context code generation for Cursor. From building megakernels that fuse an entire model's forward pass into a single launch to enabling real-time voice agents, Together AI is optimizing latency at every layer of the stack.

NVIDIA AI Infrastructure

20,738 Aufrufe • vor 2 Monaten

Realtime-VLA FLASH tackles one of the biggest deployment bottlenecks for diffusion-based VLAs: inference latency. The key idea is speculative inference for flow-matching VLAs. A lightweight draft model predicts an action chunk, while the main model’s Action Expert verifies it in parallel using flow-consistency checks instead of running full denoising every replanning round. This lets the system replace many expensive 58 ms full inference rounds with speculative rounds as fast as 7.8 ms, reducing average latency to 19.1 ms and achieving a 3.04× speedup on LIBERO while largely preserving success rate. Interesting systems insight: they profile π0 and show VLM prefill is compute-bound, while Action Denoise is memory-bound. FLASH exploits this by reusing KV cache and parallelizing verification instead of repeatedly running sequential denoising.

Robots Digest 🤖

14,576 Aufrufe • vor 4 Monaten

A 2.7x cost-per-token reduction on the same infrastructure, just from optimizing memory tiering and KV cache offload. That's the kind of number CIOs bring to the board, not a roadmap slide. At HPE Discover 2026, David Nicholson and Matthew Kimball sat down with Justin McGarry of HPE and Justin Christiansen of Intel to break down what's actually driving enterprise AI infrastructure decisions, including a new edge device running 80B parameter models from under an airplane seat.

A 2.7x cost-per-token reduction on the same infrastructure, just from optimizing memory tiering and KV cache offload. That's the kind of number CIOs bring to the board, not a roadmap slide. At HPE Discover 2026, David Nicholson and Matthew Kimball sat down with Justin McGarry of HPE and Justin Christiansen of Intel to break down what's actually driving enterprise AI infrastructure decisions, including a new edge device running 80B parameter models from under an airplane seat.

Six Five Media

100,819 Aufrufe • vor 2 Monaten

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.

Varun

37,555 Aufrufe • vor 6 Monaten

We're excited to back Sail Research, the inference platform built for long-horizon, asynchronous agents. The next wave of AI isn't a person typing into a box. It's agents running for minutes or hours, burning orders of magnitude more tokens per task. At that scale, cost and capacity beat raw latency, and teams are already hitting budget caps. Neil Movva and Samir Menon are solving this full-stack, from CUDA kernels to the API. Hear Neil and Aditya Naganath on why inference is being rebuilt for the age of agents.

We're excited to back Sail Research, the inference platform built for long-horizon, asynchronous agents. The next wave of AI isn't a person typing into a box. It's agents running for minutes or hours, burning orders of magnitude more tokens per task. At that scale, cost and capacity beat raw latency, and teams are already hitting budget caps. Neil Movva and Samir Menon are solving this full-stack, from CUDA kernels to the API. Hear Neil and Aditya Naganath on why inference is being rebuilt for the age of agents.

Kleiner Perkins

17,495 Aufrufe • vor 2 Monaten

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

We're proud to be the fastest inference provider on Artificial Analysis, OpenRouter, and Hugging Face for GLM-5.3-Flash, at 122+ TPS. All served from the US only, starting on day 0, with ZDR by default. Stay tuned for updates as our engineers continue to optimize GLM-5.3-Flash for even higher throughput and lower latency.

Baseten

23,105 Aufrufe • vor 24 Tagen

Before you ask: yes, everything is AI here. The video and sound both coming from a single text prompt using #Veo3 by Google DeepMind .Whoever is cooking the model, let him cook! Congrats matthieu lorrain and the team for the Google I/O live stream and the new Veo site!

Before you ask: yes, everything is AI here. The video and sound both coming from a single text prompt using #Veo3 by Google DeepMind .Whoever is cooking the model, let him cook! Congrats matthieu lorrain and the team for the Google I/O live stream and the new Veo site!

László Gaál

2,753,730 Aufrufe • vor 1 Jahr

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

MTS

44,057 Aufrufe • vor 4 Monaten

🚨🇨🇳 Chinese modular building revolution goes global 🔸 Modular homes and pre-fabricated ‘Holon’ buildings can be built in days rather than months 🔸 Exports rose from $1.5 billion in 2015 to $4.3 billion in 2025, reaching the US, Australia, Russia and Southeast Asia 🔸 Use of lightweight steel B-core slabs for walls improve earthquake and typhoon resistance, and is claimed to cut carbon emissions by up to 90% 🔸 Rising labor and material costs, housing shortages and long build times have increased demand from developed countries 🔸 Demand in the developing world is due to supply chain problems, lack of skilled workers and an inefficient building trade The new Chinese prefab industry is setting a new global standard for speed, efficiency and environmental performance.

🚨🇨🇳 Chinese modular building revolution goes global 🔸 Modular homes and pre-fabricated ‘Holon’ buildings can be built in days rather than months 🔸 Exports rose from $1.5 billion in 2015 to $4.3 billion in 2025, reaching the US, Australia, Russia and Southeast Asia 🔸 Use of lightweight steel B-core slabs for walls improve earthquake and typhoon resistance, and is claimed to cut carbon emissions by up to 90% 🔸 Rising labor and material costs, housing shortages and long build times have increased demand from developed countries 🔸 Demand in the developing world is due to supply chain problems, lack of skilled workers and an inefficient building trade The new Chinese prefab industry is setting a new global standard for speed, efficiency and environmental performance.

Sputnik

14,844 Aufrufe • vor 3 Monaten

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!

Andrew Ng

102,747 Aufrufe • vor 5 Monaten

Announcing our $16M Series A led by Standard Capital With participation from Y Combinator, Pioneer, Liquid 2 Ventures, Michael Grinich, Yuri Sagalov As part of this, we’re hiring for multiple roles across engineering, support and marketing ↓

Announcing our $16M Series A led by Standard Capital With participation from Y Combinator, Pioneer, Liquid 2 Ventures, Michael Grinich, Yuri Sagalov As part of this, we’re hiring for multiple roles across engineering, support and marketing ↓

Matt – Trigger.dev

81,312 Aufrufe • vor 9 Monaten

🚧 Pardon our progress as we make room for the new $82 million home of AAMU STEM. This state-of-the-art project will nearly triple the university’s instructional classroom and laboratory capacity in the areas of biology, chemistry and physics. The #AAMU150 campaign is setting a new standard for support. Join today at to be part of building the Alabama A&M of the future! #StartHere

🚧 Pardon our progress as we make room for the new $82 million home of AAMU STEM. This state-of-the-art project will nearly triple the university’s instructional classroom and laboratory capacity in the areas of biology, chemistry and physics. The #AAMU150 campaign is setting a new standard for support. Join today at to be part of building the Alabama A&M of the future! #StartHere

Alabama A&M University

12,780 Aufrufe • vor 1 Jahr

Today, we’re launching Prodigy Research (YC S26), the frontier AI trading research lab. We’re training the world’s best foundation model for quantitative finance. Our AI quant outperforms a top 10% trader at Jane Street, and we've already achieved more than 100% returns in live trading over our YC batch, while major indices were flat or down. Prodigy’s model beats Claude Fable and GPT-5.6 Sol at autonomous quant research. Our founders are brothers who have built strategies generating billions in PnL on a Jane Street trading desk, trained SoTA foundation models at Google DeepMind, and shipped cutting-edge AI to billions of users at Apple. A new era has arrived in the world of quantitative finance. To learn more, contact founders@prodigy.markets. Prodigy Research (YC S26)

Today, we’re launching Prodigy Research (YC S26), the frontier AI trading research lab. We’re training the world’s best foundation model for quantitative finance. Our AI quant outperforms a top 10% trader at Jane Street, and we've already achieved more than 100% returns in live trading over our YC batch, while major indices were flat or down. Prodigy’s model beats Claude Fable and GPT-5.6 Sol at autonomous quant research. Our founders are brothers who have built strategies generating billions in PnL on a Jane Street trading desk, trained SoTA foundation models at Google DeepMind, and shipped cutting-edge AI to billions of users at Apple. A new era has arrived in the world of quantitative finance. To learn more, contact [email protected]. Prodigy Research (YC S26)

Michael Wang

1,941,743 Aufrufe • vor 1 Monat

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Pushmeet Kohli

203,589 Aufrufe • vor 1 Jahr

We’re expanding the Gemini 3 family with the launch of Gemini 3 Flash. This model: — Combines Gemini 3’s Pro-grade reasoning with Flash-level latency, efficiency, and cost — Delivers frontier-level performance on PHD-level reasoning and knowledge benchmarks — Is our most impressive model for agentic workflows Watch Gemini 3 Flash handle hundreds of function calling options at low latency, compiling global recipes from 100 ingredients and 100 kitchen tools.

We’re expanding the Gemini 3 family with the launch of Gemini 3 Flash. This model: — Combines Gemini 3’s Pro-grade reasoning with Flash-level latency, efficiency, and cost — Delivers frontier-level performance on PHD-level reasoning and knowledge benchmarks — Is our most impressive model for agentic workflows Watch Gemini 3 Flash handle hundreds of function calling options at low latency, compiling global recipes from 100 ingredients and 100 kitchen tools.

Google AI

229,848 Aufrufe • vor 9 Monaten