Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Running a single deep coding model at max context on Cerebras requires 24 systems ($24M Capex) just to support 256 concurrent users. At that scale, $100M gets you way more memory bandwidth in standard GB300 racks.

SemiAnalysis_'s profile picture

SemiAnalysis

106,941 subscribers

93,377 Aufrufe • vor 18 Tagen •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Cerebras Code: 20x faster than Claude, 1x the price Today we are launching two monthly coding plans: ➡️Cerebras Code Pro: $50/m – for indie developers ➡️Cerebras Code Max: $200/m – for power users with 5x rate limits Both plans get: Qwen3-Coder at 2,000 tokens/s, 131K context, and no weekly limits. Sign up now:
0:43

Cerebras Code: 20x faster than Claude, 1x the price Today we are launching two monthly coding plans: ➡️Cerebras Code Pro: $50/m – for indie developers ➡️Cerebras Code Max: $200/m – for power users with 5x rate limits Both plans get: Qwen3-Coder at 2,000 tokens/s, 131K context, and no weekly limits. Sign up now:

Cerebras

461,072 Aufrufe • vor 10 Monaten

New course on serving LLMs efficiently -- how do you serve models to many concurrent users at low latency and reasonable cost? This short course is built with Red Hat and taught by Cedric Clyburn. Efficient LLM serving requires efficient memory management. A 70B-parameter model takes ~140 GB just to load the weights. On top of that, every active request needs its own chunk of GPU memory, the KV cache, to store the token context it has built up so far. In this course, you'll learn to reduce a model's memory footprint with quantization and serve it using vLLM, which handles many concurrent requests efficiently through smart memory management. Skills you'll gain: - Quantize a model and measure the accuracy tradeoff - Serve a model with vLLM and watch it handle concurrent requests efficiently - Benchmark your deployment and make informed tradeoffs between speed, cost, and accuracy Join and learn to serve LLMs efficiently:
2:11

New course on serving LLMs efficiently -- how do you serve models to many concurrent users at low latency and reasonable cost? This short course is built with Red Hat and taught by Cedric Clyburn. Efficient LLM serving requires efficient memory management. A 70B-parameter model takes ~140 GB just to load the weights. On top of that, every active request needs its own chunk of GPU memory, the KV cache, to store the token context it has built up so far. In this course, you'll learn to reduce a model's memory footprint with quantization and serve it using vLLM, which handles many concurrent requests efficiently through smart memory management. Skills you'll gain: - Quantize a model and measure the accuracy tradeoff - Serve a model with vLLM and watch it handle concurrent requests efficiently - Benchmark your deployment and make informed tradeoffs between speed, cost, and accuracy Join and learn to serve LLMs efficiently:

Andrew Ng

106,433 Aufrufe • vor 11 Tagen

DS4 running on DGX Spark (GB10 / CUDA), private branch for now. 12 tokens/sec, the memory bandwidth is limited in this system, at 270GB/sec. But prefill is ways more alighed to M3 Max at ~200 t/s. I'll release when more mature, but it is almost sure that it will get merged.
1:34

DS4 running on DGX Spark (GB10 / CUDA), private branch for now. 12 tokens/sec, the memory bandwidth is limited in this system, at 270GB/sec. But prefill is ways more alighed to M3 Max at ~200 t/s. I'll release when more mature, but it is almost sure that it will get merged.

antirez

83,448 Aufrufe • vor 1 Monat

$200/month gets you the Ferrari of vibe coding. Dozens of tool calls, instantly handled with 1.5M TPM on Cerebras Code Max available now!
0:45

$200/month gets you the Ferrari of vibe coding. Dozens of tool calls, instantly handled with 1.5M TPM on Cerebras Code Max available now!

Daniel Kim

186,185 Aufrufe • vor 8 Monaten

Cerebras just IPO’d and the stock already ran up over 100% (Save this). For the entire 70 year history of the semiconductor industry, every company on earth has followed the same process. You take a dinner plate sized silicon wafer, put hundreds of tiny chips onto it, and dice it up like a pizza. Nvidia does it this way, AMD does it this way, Intel has done it this way for six decades and everyone who tried to break that convention failed. Until Cerebras asked the most annoyingly obvious question in the industry’s history, what if you just didn’t cut it? The result is the Wafer Scale Engine, a single chip 56 times larger than Nvidia’s H100 and it fundamentally changes the physics of how AI inference works. The reason this matters is not the size, it’s the bandwidth. Every time an AI model generates a single word, it has to reach into memory, pull weights, multiply them together, and produce a prediction and when you’re running millions of concurrent sessions at once, the bottleneck is not raw processing power but how fast data moves between memory and compute. Nvidia’s H100 moves data at roughly 3 terabytes per second, while Cerebras’ WSE-3 moves data at 21 petabytes per second, roughly 7,000 times faster because memory and compute live on the same enormous piece of silicon and data barely has to travel at all. That gap is exactly why OpenAI went from 150 tokens per second on traditional GPUs to 2,000 tokens per second on Cerebras hardware, and why AWS integrated Cerebras into Bedrock to deliver roughly 5x more inference capacity in the same physical footprint. The macro setup is making the trade even more urgent. South Korea DRAM export prices recently jumped 35%, flash memory surged 47%, and SSD pricing spiked nearly 140% and every single one of those increases hits Nvidia-based infrastructure directly, because the H100 requires 80GB of the most expensive, most contested memory in the AI supply chain. Cerebras’ WSE-3 uses zero external HBM memory, baking 44GB of SRAM directly into the wafer itself which means as memory pricing goes parabolic, every CFO evaluating AI infrastructure is suddenly looking much more seriously at the architecture that sidesteps that cost entirely. The demand is already showing up in the backlog. Cerebras ended 2025 with $24.6 billion in remaining performance obligations for a company doing just over $500 million in annual revenue, that is a number that implies years of contracted growth already sitting on the books. The IPO was 20x oversubscribed, the price range was raised twice before listing, and shares opened 89% above their listing price on a $5.55 billion raise that made it the largest semiconductor IPO in history. The risks are real and worth naming. 86% of 2025 revenue came from two entities with UAE ties, U.S. revenue actually fell 34% to $187 million, and the $20 billion OpenAI contract is conditional, if Cerebras misses delivery milestones, OpenAI can terminate and trigger repayment demands on a $1 billion loan facility. And yet the market is valuing Cerebras at roughly 91x trailing revenue, richer than Nvidia, AMD, and Arm combined. What investors are betting on is not that Cerebras beats Nvidia, it is that the inference supercycle is large enough to support an entirely different architecture optimized for a different workload, and that $24.6 billion in contracted backlog converts to diversified revenue before the market starts asking harder questions. CEO Andrew Feldman said this took a decade of late nights to get right, everyone who tried to copy it failed and given that the entire inference economy is now running through exactly the bottleneck Cerebras was built to eliminate, the market is starting to believe him.
2:04

Cerebras just IPO’d and the stock already ran up over 100% (Save this). For the entire 70 year history of the semiconductor industry, every company on earth has followed the same process. You take a dinner plate sized silicon wafer, put hundreds of tiny chips onto it, and dice it up like a pizza. Nvidia does it this way, AMD does it this way, Intel has done it this way for six decades and everyone who tried to break that convention failed. Until Cerebras asked the most annoyingly obvious question in the industry’s history, what if you just didn’t cut it? The result is the Wafer Scale Engine, a single chip 56 times larger than Nvidia’s H100 and it fundamentally changes the physics of how AI inference works. The reason this matters is not the size, it’s the bandwidth. Every time an AI model generates a single word, it has to reach into memory, pull weights, multiply them together, and produce a prediction and when you’re running millions of concurrent sessions at once, the bottleneck is not raw processing power but how fast data moves between memory and compute. Nvidia’s H100 moves data at roughly 3 terabytes per second, while Cerebras’ WSE-3 moves data at 21 petabytes per second, roughly 7,000 times faster because memory and compute live on the same enormous piece of silicon and data barely has to travel at all. That gap is exactly why OpenAI went from 150 tokens per second on traditional GPUs to 2,000 tokens per second on Cerebras hardware, and why AWS integrated Cerebras into Bedrock to deliver roughly 5x more inference capacity in the same physical footprint. The macro setup is making the trade even more urgent. South Korea DRAM export prices recently jumped 35%, flash memory surged 47%, and SSD pricing spiked nearly 140% and every single one of those increases hits Nvidia-based infrastructure directly, because the H100 requires 80GB of the most expensive, most contested memory in the AI supply chain. Cerebras’ WSE-3 uses zero external HBM memory, baking 44GB of SRAM directly into the wafer itself which means as memory pricing goes parabolic, every CFO evaluating AI infrastructure is suddenly looking much more seriously at the architecture that sidesteps that cost entirely. The demand is already showing up in the backlog. Cerebras ended 2025 with $24.6 billion in remaining performance obligations for a company doing just over $500 million in annual revenue, that is a number that implies years of contracted growth already sitting on the books. The IPO was 20x oversubscribed, the price range was raised twice before listing, and shares opened 89% above their listing price on a $5.55 billion raise that made it the largest semiconductor IPO in history. The risks are real and worth naming. 86% of 2025 revenue came from two entities with UAE ties, U.S. revenue actually fell 34% to $187 million, and the $20 billion OpenAI contract is conditional, if Cerebras misses delivery milestones, OpenAI can terminate and trigger repayment demands on a $1 billion loan facility. And yet the market is valuing Cerebras at roughly 91x trailing revenue, richer than Nvidia, AMD, and Arm combined. What investors are betting on is not that Cerebras beats Nvidia, it is that the inference supercycle is large enough to support an entirely different architecture optimized for a different workload, and that $24.6 billion in contracted backlog converts to diversified revenue before the market starts asking harder questions. CEO Andrew Feldman said this took a decade of late nights to get right, everyone who tried to copy it failed and given that the entire inference economy is now running through exactly the bottleneck Cerebras was built to eliminate, the market is starting to believe him.

Milk Road AI

30,441 Aufrufe • vor 1 Monat

Pipe (Pipe Network) is a decentralized content delivery network (dCDN) that accelerates delivery by leveraging users’ excess storage & bandwidth. More nodes = faster delivery. More from David Rhodus 🇺🇸 at Breakpoint '24
3:06

Pipe (Pipe Network) is a decentralized content delivery network (dCDN) that accelerates delivery by leveraging users’ excess storage & bandwidth. More nodes = faster delivery. More from David Rhodus 🇺🇸 at Breakpoint '24

Solana

78,975 Aufrufe • vor 1 Jahr

Max Thomas 🇺🇸 running a PB of 9.90s (1.2) in his 100m season opener at the Florida Relays!
0:34

Max Thomas 🇺🇸 running a PB of 9.90s (1.2) in his 100m season opener at the Florida Relays!

Track & Field Gazette

29,538 Aufrufe • vor 2 Monaten

With TRAE Max Mode you can now scale your context window from 200k → 1M for deep coding, longer reasoning, and complex projects. You can stay in control with full transparency on token usage, cost, and fast requests right after each conversation.
0:28

With TRAE Max Mode you can now scale your context window from 200k → 1M for deep coding, longer reasoning, and complex projects. You can stay in control with full transparency on token usage, cost, and fast requests right after each conversation.

TRAE

14,402 Aufrufe • vor 8 Monaten

Perplexity Introduces Personalized Memory 🔥 TLDR : - Gives AI assistants memory to recall preferences, interests, and past chats. - It removes manual context engineering by auto loading relevant context. - Answers become more accurate and personalized using direct memory retrieval. - Users stay fully in control with on or off switches, incognito protection, and encryption. - Memory works across all models so your context follows you anywhere in Perplexity. - Comet Assistant also gets true memory, making it more consistent, personalized, and powerful.
0:30

Perplexity Introduces Personalized Memory 🔥 TLDR : - Gives AI assistants memory to recall preferences, interests, and past chats. - It removes manual context engineering by auto loading relevant context. - Answers become more accurate and personalized using direct memory retrieval. - Users stay fully in control with on or off switches, incognito protection, and encryption. - Memory works across all models so your context follows you anywhere in Perplexity. - Comet Assistant also gets true memory, making it more consistent, personalized, and powerful.

AshutoshShrivastava

22,006 Aufrufe • vor 6 Monaten

The first-ever hydrogen-powered NVIDIA GB300 NVL72 systems are online. Training & inference at scale — with zero emissions, zero water. Lambda × Supermicro × ECL →
0:39

The first-ever hydrogen-powered NVIDIA GB300 NVL72 systems are online. Training & inference at scale — with zero emissions, zero water. Lambda × Supermicro × ECL →

Lambda

17,276 Aufrufe • vor 8 Monaten

🎁 We're giving away 5 Windsurf plans ($250 credit each)! Try SWE-1.6 — Cognition’s latest fast and intelligent agentic coding model, powered by Cerebras. In a side-by-side with Claude, the speed difference is clear. More iterations, faster fixes, better code. 💬Comment why you want access to enter. Five winners will be selected at random within 48 hours.
1:08

🎁 We're giving away 5 Windsurf plans ($250 credit each)! Try SWE-1.6 — Cognition’s latest fast and intelligent agentic coding model, powered by Cerebras. In a side-by-side with Claude, the speed difference is clear. More iterations, faster fixes, better code. 💬Comment why you want access to enter. Five winners will be selected at random within 48 hours.

Cerebras

104,494 Aufrufe • vor 1 Monat

🚨 Cerebras Inference is now 3x faster: Llama3.1-70B just broke 2,100 tokens/s - 16x faster than the fastest GPU solution - 8x faster than GPUs running Llama *3B* - It's like the perf of a new hardware generation in a single software release Available now at
0:26

🚨 Cerebras Inference is now 3x faster: Llama3.1-70B just broke 2,100 tokens/s - 16x faster than the fastest GPU solution - 8x faster than GPUs running Llama *3B* - It's like the perf of a new hardware generation in a single software release Available now at

Cerebras

235,976 Aufrufe • vor 1 Jahr

MiniMax M3 support added to mlx-vlm with MSA implementation! 🚀 Tested on M3 Ultra 512GB running at 24 tps with peak memory ~240GB. Now working on optimizing performance and adding ton of tests 💪 Model is here: PR is here:
0:24

MiniMax M3 support added to mlx-vlm with MSA implementation! 🚀 Tested on M3 Ultra 512GB running at 24 tps with peak memory ~240GB. Now working on optimizing performance and adding ton of tests 💪 Model is here: PR is here:

Ivan Fioravanti ᯅ

23,587 Aufrufe • vor 2 Tagen

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”
1:31

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,827 Aufrufe • vor 27 Tagen

Built FlickAI in 24 hours at the Cerebras x Cline Vibe Coder Hackathon ▨ A desktop AI assistant that: • Sees what’s on your screen • Wakes up instantly • Helps with coding, emails, notes anything and everything. Built this with my teammate samarth saxena
1:31

Built FlickAI in 24 hours at the Cerebras x Cline Vibe Coder Hackathon ▨ A desktop AI assistant that: • Sees what’s on your screen • Wakes up instantly • Helps with coding, emails, notes anything and everything. Built this with my teammate samarth saxena

Maaz (testnet arc)

27,129 Aufrufe • vor 4 Monaten

AI coding without systems thinking is just tech debt on speedrun. Delty (@delty_ai) is an AI Staff Engineer for your team. With deep expertise, it designs software systems, evaluates tradeoffs and makes AI coding agents smarter with your engineering context. Congrats on the launch, Lalit Kundu and Catherine!
1:09

AI coding without systems thinking is just tech debt on speedrun. Delty (@delty_ai) is an AI Staff Engineer for your team. With deep expertise, it designs software systems, evaluates tradeoffs and makes AI coding agents smarter with your engineering context. Congrats on the launch, Lalit Kundu and Catherine!

Y Combinator

73,297 Aufrufe • vor 1 Jahr

Draft email replies faster. Suggested Replies offer one-tap responses that are based on the context of the conversation and match how you write. 🪄 Suggested Replies are available to US users at no cost! Proofread requires a Google AI Pro or Ultra subscription. →
0:28

Draft email replies faster. Suggested Replies offer one-tap responses that are based on the context of the conversation and match how you write. 🪄 Suggested Replies are available to US users at no cost! Proofread requires a Google AI Pro or Ultra subscription. →

Gmail

22,253 Aufrufe • vor 4 Monaten

AI coding agents hit a wall when codebases get massive. Even with 2M token context windows, a 10M line codebase needs 100M tokens. The real bottleneck isn't just ingesting code - it's getting models to actually pay attention to all that context effectively.
1:30

AI coding agents hit a wall when codebases get massive. Even with 2M token context windows, a 10M line codebase needs 100M tokens. The real bottleneck isn't just ingesting code - it's getting models to actually pay attention to all that context effectively.

Garry Tan

976,161 Aufrufe • vor 1 Jahr

Elon Musk: In order to run like really deep intelligence, you need a lot of compute. It’s not like you can just fire up a PC in your basement and be running AGI. At least not yet. Grok was trained on eight thousand A100s running at peak efficiency. And Grok’s going to get a lot better, by the way.
1:28

Elon Musk: In order to run like really deep intelligence, you need a lot of compute. It’s not like you can just fire up a PC in your basement and be running AGI. At least not yet. Grok was trained on eight thousand A100s running at peak efficiency. And Grok’s going to get a lot better, by the way.

Ian Miles Cheong

85,675 Aufrufe • vor 7 Monaten

OpenAI Codex-Spark powered by Cerebras You can now just build things faster—at 1,000 tokens/s.
0:47

OpenAI Codex-Spark powered by Cerebras You can now just build things faster—at 1,000 tokens/s.

Cerebras

287,287 Aufrufe • vor 4 Monaten