SemiAnalysis
@SemiAnalysis_ • 175,882 subscribers
Shorts
Videos
GLM 5.3’s biggest leap came from post-training alone. “GLM 5.3 was built entirely by post-training the previous model. For me, that was a wake-up call.” “Post-training is far more GPU-efficient than pre-training from scratch. Plenty of companies can’t afford to train a model from scratch, but they have proprietary data. They can post-train themselves and beat some of the closed models.” “There’s still so much juice left to squeeze from post-training. We compared GLM 5.3’s cyber capabilities with 5.2, and it’s a night and day difference. That’s all just post-training.”
SemiAnalysis185,593 Aufrufe • vor 1 Tag
We cut open the Kirin 9030 and put it under an electron microscope. The smallest metal pitch measures 32.5 nanometers. That is tighter than Intel 18A, their brand new leading edge node. A Chinese fab with no EUV is out-pitching Intel's EUV node by roughly 10 percent. What's going on here? 🤯 "This is the HiSilicon Kirin 9030, the chip inside Huawei's newest flagship phone. A few weeks ago, we cut it open, put it under an electron microscope, and measured the smallest wires inside the chip, the metal pitch. And what we saw was unexpected. The smallest metal pitch inside the Kirin 9030 measures only 32.5 nanometers. That’s smaller than the metal pitch in Panther Lake, which is based on Intel's brand-new 18A node. A Chinese fab, cut off from the most advanced tools, without EUV, is packing its wires about ten percent tighter than Intel's leading edge EUV node. What’s going on here?"
SemiAnalysis1,017,918 Aufrufe • vor 3 Monaten
China’s frontier models already run on China’s own chips. “Remember, the GLM-5 paper is almost a year old at this point. They say: ‘From the first day, GLM-5 is full-stack adapted to Chinese GPU ecosystems. We have successfully completed deep optimization, spanning from underlying kernels to upper-level inference frameworks, across seven mainstream domestic chip platforms, including Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX and Enflame.’” “It’s not necessarily NVIDIA GPUs and kernels we’re talking about. They’re serving this model, at least they claim, for many months on seven different domestic Chinese accelerators.” “People thought it was impossible that it’s served on GPUs by a Chinese lab, because Chinese labs aren’t that GPU rich. But turns out they’re using Chinese hardware. They plumb through everything, the kernels, the systems, while serving those models efficiently.”
SemiAnalysis37,009 Aufrufe • vor 3 Tagen
Sparse attention fixes memory bandwidth, not memory capacity. “The whole idea of HiCache is trying to offload KV cache from the HBM to a lower tier, higher capacity but lower bandwidth memory, and only load the most relevant set of tokens during computation.” “Sparse attention doesn’t mitigate the memory capacity bottleneck. It’s a technique that helps a lot with memory bandwidth.” “During the sparse attention computation, you still need to select the tokens. So you still have to keep all the tokens in context, and you cannot discard them. At some point, it still has to be loaded into HBM.” “That’s where HiSparse becomes useful, because it can apply the typical caching rules so that you could load the most relevant parts into HBM.”
SemiAnalysis24,911 Aufrufe • vor 2 Tagen
Even the frontier labs underestimated their own demand. “What gets confusing to me is how nobody forecast this.” “Even the frontier labs, who really understand how these models work, did not expect to be this profitable, to serve this many users and to grow this fast.” “They had to do crazy things and sign big contracts at very high premiums just to get access to GPUs tomorrow to serve their users.” “I don’t view this as an AI fast takeoff. It seems to be a revenue and profitability fast takeoff for a lot of these labs.”
SemiAnalysis50,682 Aufrufe • vor 7 Tagen
"Google has an L culture and has never actually made anything internally. They've only ever acquired all their innovation, and their execution is very poor." "Let's seriously do some accounting. Everything other than Core Search has been purchased. YouTube, AdMob, DoubleClick, AdSense, Google Maps, Android, all of it." "IBM had effectively 90% market share in 1950. They said no, we're not gonna do PC, we're gonna focus on mainframe. And then they proceeded to lose." "I feel like today, more than any other point, Google has that opportunity. They're gonna have a good enough product on the Transformer, but they're probably not going to stay for the next, and the next, and the next. And then they'll quietly bow out." "When we look back in five, ten, twenty years at what was the moment things started to change, it's like right now." — Doug O'Laughlin
SemiAnalysis495,714 Aufrufe • vor 2 Monaten
Is GLM 5.3 worth sounding the alarms? 🚨 “As long as you have the budget, it will write faster kernels than humans.” Kimbo (Kimbo) and Wenyao (Wenyao Gao) join Jordan (Jordan Nanos) this week to unpack sparse attention, the memory bottleneck nobody’s pricing in, and why cheaper tokens might mean more HBM, not less.
SemiAnalysis27,154 Aufrufe • vor 5 Tagen
The AI buildout needs more capital than backstops can cover. “There’s bank lenders, insurance lenders, they don’t want to take a lot of risk. You’ve got private equity, they want to take a lot of risk. You’ve got private credit, they’re in the middle.” “What you do is you tranche risk. If there are losses, the equity will take the loss first, then the B piece, and then part of the A2. The loss has to be so severe to affect the senior tranche, which is how you can create different risk profiles.” “We think NVIDIA has the capacity to support up to 46 gigawatts. That’s still not enough. There’s 240 gigawatts that’s gonna come in.” “The lending market has to evolve to take risks outside of backstops. It’s really the biggest question of our time.”
SemiAnalysis37,148 Aufrufe • vor 8 Tagen
GPT-6 Astra reportedly gets deeper without getting bigger. The labs already know what that means for scaling. "It's basically confirmed that GPT-6 Astra uses loop transformers, which means that instead of adding parameter count, it goes through the layers more than once. You add compute depth, but you don't increase the size of the model." "The labs are probably the people that are best positioned to say which way models are scaling. I think that's a tell that they're not seeing parameter sizes scaling as aggressively in their roadmaps, in what they find in their research."
SemiAnalysis123,781 Aufrufe • vor 1 Monat
Ep. 027 - OpenAI Jalapeño: Better Than Nvidia Blackwell (Accelerators) This week Bryan, Myron and Jordan (Jordan Nanos) discuss our recent article on OpenAI Jalapeño. They cover the performance, architecture, programming model, implications for NVIDIA and more. 0:00 Cold Open 1:05 Jalapeno Overview 4:22 Tokens Per Megawatt 9:15 Benchmark Caveats 13:48 The CUDA Moat 21:12 How OpenAI Did It 32:13 Samsung HBM4 42:10 AI-Designed Silicon 49:24 Architecture Deep Dive 56:58 Doom and Wrap
SemiAnalysis175,009 Aufrufe • vor 1 Monat
$7 trillion is about to get spent on GPU clouds, and every provider is being asked to get 100x bigger overnight. ClusterMAX 3.0 spent three months finding out who actually can. Ep. 033 of SemiAnalysis Weekly is live. Jordan Nanos (Jordan Nanos) sits down with Sam Harshe (sharshe) and Pratt Bhatt (Pratt) fresh off testing 77 GPU clouds: the new rankings, health checks, cache hit rates, NCCL, agents on clusters, a Linux kernel bug, security, financing and backstops, H100s, and hosted RL training. "It's my first time seeing sunlight today, actually, since testing began." "This time we rank 77 providers." "Just about everyone in the industry looks like a genius right now." "No way. I thought you could just earn 80% margins on the inference business, right?" "Who are you going to trust for your Vera Rubin going into next year?"
SemiAnalysis91,077 Aufrufe • vor 25 Tagen
Open-source models are already taking the easier tasks. The bull case for frontier labs rests on one question. "A lot of software engineering, white-collar work in general, does not need Fable 5.1 or Astra 6 level intelligence." "A lot of businesses, especially lower gross margin businesses, are very rationally making the decision to start offloading some of the easier tasks to these increasingly capable open-source models." "So whether the frontier labs are a durable business model really comes down to your belief on whether the TAM of new things you can do with increasingly smarter frontier intelligence will outgrow the portion of tasks being offloaded to these cheaper models." "If you were to magically spawn 100 million superintelligent PhDs who are experts in every field and don't need to eat or sleep, do you think the economy would be able to absorb that relatively quickly and still get high ROI?"
SemiAnalysis39,244 Aufrufe • vor 10 Tagen
Neocloud is now an entire industry. The word was coined by a SemiAnalysis researcher. Some of the companies using it still have no idea. "Nobody believes me when I say that. I'll tell people externally, yeah, we're the ones who invented the word, and they're like, No, you didn't. I'm like, Yeah, great." "This guy didn't know us at all. He was like, Cool man, what do you guys do? I'm like, What the f**k? We invented your entire industry."
SemiAnalysis97,280 Aufrufe • vor 28 Tagen
"It's like a zombie apocalypse, but you don't know how many waves of zombies are coming." H100 rental prices went up, on a GPU that should be sliding down its depreciation curve. That curve is what lenders underwrite, and there's $7 trillion of debt coming by 2029. Daniel Nishball kicks off our new Compute Capital Markets group with Jordan Nanos. Nvidia's backstops, ClusterMAX as a credit signal, whether each token is profitable and more.
SemiAnalysis32,743 Aufrufe • vor 9 Tagen
A $200 AI subscription can be worth $12,000 in tokens. Which plan delivers it is not close. "They're selling to willing buyers at the fair market price. This is capitalism." "You can pay $200 for a Claude plan and get $12,000 of Opus 5.5 tokens at API pricing. So that's obviously a good deal." "You have to say the $200 plan, when running a particular workload on a particular model, gives you some dollar amount. But then you also have 5.5, which is a beast. $6K or $5,000 worth of value." "And when you do that comparison, it's not even a question of who's providing more value. ChatGPT versus Anthropic subscriptions." Max Kan and Andrew Megalaa join Jordan Nanos this week
SemiAnalysis37,697 Aufrufe • vor 11 Tagen
The fastest finish is not always the most efficient one. “From personal experience trying out different models, you can definitely see token efficiency on the OpenAI side. But it’s what I like to think of as short-term token efficiency.” “Yes, I’ll get this immediate task done, but there are so many follow-ups that Fable would have just done, and Astra might have just not done for the sake of completing the task faster.” “In the long term, I feel like the Anthropic models build you the better codebase, and definitely a more readable one. If you read some of these Astra tests, it’s something else.” “I think models matter quite a bit for token efficiency, over which user is actually using them.”
SemiAnalysis25,716 Aufrufe • vor 8 Tagen
Jim Cramer "There is a company that I regard as the gospel, SemiAnalysis. I don't think people realize that SemiAnalysis is the arbiter. They're like God in the semis, and when they do, when they bless something. They are the most honest guys I've come across"
SemiAnalysis513,175 Aufrufe • vor 6 Monaten
ALERT🚨🚨: On apples-to-apples InferenceX Comparisons, TPUv7 Ironwood achieves 50% better perf per dollar than Blackwell Ultra on the new TorchTPU external inference stack! TorchTPU brings native PyTorch to TPUs, & along with Google open-sourcing a bunch of their Pallas inference kernels, Google has laid out a solid foundation for TPU to rapidly externalize. We at SemiAnalysis strongly believe that TPU externalization is heading in the right direction and moving full steam ahead.
SemiAnalysis105,257 Aufrufe • vor 1 Monat