SemiAnalysis's banner

SemiAnalysis

@SemiAnalysis_ • 93,698 subscribers

Shorts

hi i'm semiman

hi i'm semiman

51,782 просмотров

Jensen giving out Pocky sticks in Asia. Just like how some neoclouds feel that NVIDIA gives brownie points to neoclouds that are marketing or selling NVIDIA software that no AI-native end user wants, like MIG, NIMS, and NVAIE. Some neoclouds feel that NVIDIA gives brownie points to them for buying NVIDIA BlueField-3 DPUs even though they are nowhere near having the skillset to properly use them.

Jensen giving out Pocky sticks in Asia. Just like how some neoclouds feel that NVIDIA gives brownie points to neoclouds that are marketing or selling NVIDIA software that no AI-native end user wants, like MIG, NIMS, and NVAIE. Some neoclouds feel that NVIDIA gives brownie points to them for buying NVIDIA BlueField-3 DPUs even though they are nowhere near having the skillset to properly use them.

302,452 просмотров

TSMC 2nm process data leak investigation🧵 Taipei time August 5th, a trade secret leak case involving TSMC's 2nm technology was uncovered. 9 engineers were suspected of leaking confidential information, and 3 of them were detained by the Intellectual Property and Commercial Court.

TSMC 2nm process data leak investigation🧵 Taipei time August 5th, a trade secret leak case involving TSMC's 2nm technology was uncovered. 9 engineers were suspected of leaking confidential information, and 3 of them were detained by the Intellectual Property and Commercial Court.

351,136 просмотров

A Day in the Life of Dwarkesh Patel's Cousin

A Day in the Life of Dwarkesh Patel's Cousin

50,137 просмотров

Congrats to Dylan's Cousin, Dwarkesh Patel , on reaching 1 million subscribers on YouTube! Dylan & Dwarkesh Patel's family come from Gujarat but they grew up in America most of their lives mainly in Econo Lodge motels.

Congrats to Dylan's Cousin, Dwarkesh Patel , on reaching 1 million subscribers on YouTube! Dylan & Dwarkesh Patel's family come from Gujarat but they grew up in America most of their lives mainly in Econo Lodge motels.

112,546 просмотров

This is how OpenAI comes back and beats Anthropic

This is how OpenAI comes back and beats Anthropic

41,939 просмотров

AMD's software quality has massively improved since AMD DC GPU division went hardcore mode back in January 2025. It isn't just us saying this but many of AMD's Instinct GPU customers are saying this too. Great work to Anush Elangovan 's team of amazing engineers.🥳

AMD's software quality has massively improved since AMD DC GPU division went hardcore mode back in January 2025. It isn't just us saying this but many of AMD's Instinct GPU customers are saying this too. Great work to Anush Elangovan 's team of amazing engineers.🥳

65,099 просмотров

AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's Anush Elangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's Anush Elangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

53,783 просмотров

A lot of AMD's most cracked engineers that work on the latest disagg inferencing & MoRI first princples collective library are in China celebrating major chinese holidays. Should we get human Anush Elangovan in an chinese dance battle verus AI?

A lot of AMD's most cracked engineers that work on the latest disagg inferencing & MoRI first princples collective library are in China celebrating major chinese holidays. Should we get human Anush Elangovan in an chinese dance battle verus AI?

21,470 просмотров

Something NVIDIA & Google do better than anyone else is software-hardware-system co-design, and not just optimizing hardware for current model architectures, but predicting future ones. Back in early 2022, when NVIDIA started the design process for NVL72, MoE (Mixture of Experts) models were not yet the standard, and dense models were still dominant for frontier models. However, NVIDIA's strong software-hardware co-design culture enabled them to make a calculated bet that MoEs were the future, and they built NVL72 specifically for best MoE performance per TCO (Total Cost of Ownership). Furthermore, back in 2022, disaggregated prefill and wide expert parallelism (wideEP) MoE inference optimizations hadn't been invented yet, but it turns out that these MoE inference optimizations work best on large-scale systems like NVL72. While most other AI chip companies' in-house AI labs focus on training small 5B models that mainly use data parallelism, NVIDIA and Google's in-house AI labs continuously push the boundaries of model architecture and training recipes, such as NVFP4 training. Just like Super Idol & IShowSpeed, there must be a strong partnership between software engineers and hardware engineers to deliver the best systems that maximize performance per TCO.

Something NVIDIA & Google do better than anyone else is software-hardware-system co-design, and not just optimizing hardware for current model architectures, but predicting future ones. Back in early 2022, when NVIDIA started the design process for NVL72, MoE (Mixture of Experts) models were not yet the standard, and dense models were still dominant for frontier models. However, NVIDIA's strong software-hardware co-design culture enabled them to make a calculated bet that MoEs were the future, and they built NVL72 specifically for best MoE performance per TCO (Total Cost of Ownership). Furthermore, back in 2022, disaggregated prefill and wide expert parallelism (wideEP) MoE inference optimizations hadn't been invented yet, but it turns out that these MoE inference optimizations work best on large-scale systems like NVL72. While most other AI chip companies' in-house AI labs focus on training small 5B models that mainly use data parallelism, NVIDIA and Google's in-house AI labs continuously push the boundaries of model architecture and training recipes, such as NVFP4 training. Just like Super Idol & IShowSpeed, there must be a strong partnership between software engineers and hardware engineers to deliver the best systems that maximize performance per TCO.

51,021 просмотров

Racks on racks on racks Racks on Racks, Buildin' Stacks Stackin' GPUs Power Whipping Liquid Flex Drip Check

Racks on racks on racks Racks on Racks, Buildin' Stacks Stackin' GPUs Power Whipping Liquid Flex Drip Check

68,561 просмотров

In late 2023, AMD made its best acquisition to date: NodAI, led by CEO Anush Elangovan. At the time, AMD had a 0% chance of challenging CUDA, while AMD was strong in hardware, it didn't understand software. Since the NodAI acquisition, Anush has driven AMD’s AI software strategy and helped reshape the org around the importance of software and software–hardware co-design. As a result, AMD now has a non-zero chance of breaking the CUDA moat. Had NVIDIA acquired NodAI instead, AMD would almost certainly still be stuck at a 0% chance.

In late 2023, AMD made its best acquisition to date: NodAI, led by CEO Anush Elangovan. At the time, AMD had a 0% chance of challenging CUDA, while AMD was strong in hardware, it didn't understand software. Since the NodAI acquisition, Anush has driven AMD’s AI software strategy and helped reshape the org around the importance of software and software–hardware co-design. As a result, AMD now has a non-zero chance of breaking the CUDA moat. Had NVIDIA acquired NodAI instead, AMD would almost certainly still be stuck at a 0% chance.

39,741 просмотров

MI355 disaggregated serving is competitive to B200 disaggregated serving for FP8 but when composability all the optimizations that frontier labs use together like wide expert parallelism+disagg+FP4+kvcache offloading AMD is still jestermaxxing & lackluster. King Anush Elangovan needs to focus on composability of inference optimizations for ROCm stack in order to defeat the CUDA moat

MI355 disaggregated serving is competitive to B200 disaggregated serving for FP8 but when composability all the optimizations that frontier labs use together like wide expert parallelism+disagg+FP4+kvcache offloading AMD is still jestermaxxing & lackluster. King Anush Elangovan needs to focus on composability of inference optimizations for ROCm stack in order to defeat the CUDA moat

22,416 просмотров

Since RCCL is an fork of NCCL, RCCL is basically a copy+paste carbon copy of NCCL except it takes months for new features added to NCCL to reach RCCL. This is clearly not optimal for AMD to be building on their competitors platform as that means that AMD will never be better than NVIDIA or even reach parity (when iso-time) due to the delays/engineering burden of syncing with upstream. AMD is working on an moonshot project cuz MORI-CCL aims to be an first principles from scratch re-built of the AMD collective library software to not be dependent of their competitor's software. Ironically enough with MORI-CCL, it currently doesn't support AMD's Pensando NICs yet it supports NVIDIA ConnectX-7 NICs with AMD GPUs. Support for AMD's Pensando NICs is coming after support for NVIDIA ConnectX-7 NICs.

Since RCCL is an fork of NCCL, RCCL is basically a copy+paste carbon copy of NCCL except it takes months for new features added to NCCL to reach RCCL. This is clearly not optimal for AMD to be building on their competitors platform as that means that AMD will never be better than NVIDIA or even reach parity (when iso-time) due to the delays/engineering burden of syncing with upstream. AMD is working on an moonshot project cuz MORI-CCL aims to be an first principles from scratch re-built of the AMD collective library software to not be dependent of their competitor's software. Ironically enough with MORI-CCL, it currently doesn't support AMD's Pensando NICs yet it supports NVIDIA ConnectX-7 NICs with AMD GPUs. Support for AMD's Pensando NICs is coming after support for NVIDIA ConnectX-7 NICs.

33,430 просмотров

We have been cookin with Seedance-2 🤤

We have been cookin with Seedance-2 🤤

19,502 просмотров

NVIDIA's CUDA Moat is becoming a Copper Moat with scale-up networking. Bigger models demand more from the scale-up network, and we❤️ Copper

NVIDIA's CUDA Moat is becoming a Copper Moat with scale-up networking. Bigger models demand more from the scale-up network, and we❤️ Copper

38,731 просмотров

Choose your fighter 😈

Choose your fighter 😈

15,389 просмотров

Peter Griffin explains Elon Musk 's first principle move when Tennessee pushed back on gas turbines for his AI datacenter, so Elon just built a gigawatt gas turbines across the border in Mississippi. Now Colossus-2 will be the first to 1GW

Peter Griffin explains Elon Musk 's first principle move when Tennessee pushed back on gas turbines for his AI datacenter, so Elon just built a gigawatt gas turbines across the border in Mississippi. Now Colossus-2 will be the first to 1GW

26,590 просмотров

Ai Accelerator Chips

Ai Accelerator Chips

26,198 просмотров

"Speed in The Moat" - Anush Elangovan Last year, AMD software is NP hard to use. After Anush & Lisa 10x the sense of urgency, the software is way better.

"Speed in The Moat" - Anush Elangovan Last year, AMD software is NP hard to use. After Anush & Lisa 10x the sense of urgency, the software is way better.

17,072 просмотров

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Running a single deep coding model at max context on Cerebras requires 24 systems ($24M Capex) just to support 256 concurrent users. At that scale, $100M gets you way more memory bandwidth in standard GB300 racks.

Running a single deep coding model at max context on Cerebras requires 24 systems ($24M Capex) just to support 256 concurrent users. At that scale, $100M gets you way more memory bandwidth in standard GB300 racks.

92,797 просмотров • 6 дней назад

Cerebras represents a whole NVL72 rack on a single wafer. By routing around defects and staying on-die, they bypass the networking power bottleneck that traditional GPU clusters face.

Cerebras represents a whole NVL72 rack on a single wafer. By routing around defects and staying on-die, they bypass the networking power bottleneck that traditional GPU clusters face.

81,668 просмотров • 10 дней назад

Jim Cramer "There is a company that I regard as the gospel, SemiAnalysis. I don't think people realize that SemiAnalysis is the arbiter. They're like God in the semis, and when they do, when they bless something. They are the most honest guys I've come across"

Jim Cramer "There is a company that I regard as the gospel, SemiAnalysis. I don't think people realize that SemiAnalysis is the arbiter. They're like God in the semis, and when they do, when they bless something. They are the most honest guys I've come across"

512,640 просмотров • 2 месяцев назад

The next-gen Cerebras CS4 is staying on 5nm. Why? Because going to 3nm doesn't magically fix the fact that SRAM scaling has completely flattened.

The next-gen Cerebras CS4 is staying on 5nm. Why? Because going to 3nm doesn't magically fix the fact that SRAM scaling has completely flattened.

54,796 просмотров • 8 дней назад

After studying 300 Leetcode Hards, solving every Jane Street puzzle from the Dwarkesh ads, and watching one Horace He lecture, he finally landed the $400k annualized Jane Street internship. Unfortunately, during onboarding his manager said “this diff is negative alpha,” so Jane Street deployed an AI model to translate all feedback into HR-safe speech in real time.

After studying 300 Leetcode Hards, solving every Jane Street puzzle from the Dwarkesh ads, and watching one Horace He lecture, he finally landed the $400k annualized Jane Street internship. Unfortunately, during onboarding his manager said “this diff is negative alpha,” so Jane Street deployed an AI model to translate all feedback into HR-safe speech in real time.

134,572 просмотров • 23 дней назад

to be clear, NVIDIA is NOT a car

to be clear, NVIDIA is NOT a car

280,900 просмотров • 1 месяц назад

Anthropic may have built themselves into an innovator's dilemma with Claude's CLI focus while the real AI agent revolution needs something much bigger.

Anthropic may have built themselves into an innovator's dilemma with Claude's CLI focus while the real AI agent revolution needs something much bigger.

77,283 просмотров • 18 дней назад

Technical breakdown of tokenizer improvements from GPT 4.6 to 4.7

Technical breakdown of tokenizer improvements from GPT 4.6 to 4.7

37,784 просмотров • 18 дней назад

Long-term memory agreements have historically been the sign of the top. Micron prints incredible earnings, stock sells off. Everyone rotates out. But the prepayment terms and pricing floors this cycle look nothing like prior rounds. Our Core Research team breaks down why they're still bullish.

Long-term memory agreements have historically been the sign of the top. Micron prints incredible earnings, stock sells off. Everyone rotates out. But the prepayment terms and pricing floors this cycle look nothing like prior rounds. Our Core Research team breaks down why they're still bullish.

95,485 просмотров • 2 месяцев назад

It has become extremely trendy among some SF AI researchers to donate to shrimp welfare. They estimate that they help improve the welfare of 1,500 shrimps per year for every $1 donated. Why do they donate to shrimps? They claim that it is the most cost-effective way of reducing suffering of sentient beings. Note that the Shrimp Welfare non-profit does not actually prevent shrimps from being killed but instead promotes the use of electrical stunning as a more humane slaughtering method that aligns with the goal of reducing shrimp suffering.

It has become extremely trendy among some SF AI researchers to donate to shrimp welfare. They estimate that they help improve the welfare of 1,500 shrimps per year for every $1 donated. Why do they donate to shrimps? They claim that it is the most cost-effective way of reducing suffering of sentient beings. Note that the Shrimp Welfare non-profit does not actually prevent shrimps from being killed but instead promotes the use of electrical stunning as a more humane slaughtering method that aligns with the goal of reducing shrimp suffering.

264,319 просмотров • 6 месяцев назад

Is OpenAI actually back? Jordan Nanos, @Dylan522p, @FabricatedKnowledge, and Max Kan break down whether OpenAI has truly recovered from their recent struggles and what this means for the AI race.

Is OpenAI actually back? Jordan Nanos, @Dylan522p, @FabricatedKnowledge, and Max Kan break down whether OpenAI has truly recovered from their recent struggles and what this means for the AI race.

25,187 просмотров • 16 дней назад

can i get a KA-CHOW!

can i get a KA-CHOW!

65,289 просмотров • 1 месяц назад

Here's what nobody is telling you about memory companies: hedge funds with billion dollar positions are flying blind until earnings calls reveal demand collapsed. Smart money is paying for real-time shipment data from Korea, Taiwan, China because getting caught in a cycle turn costs more than the tracking fee.

Here's what nobody is telling you about memory companies: hedge funds with billion dollar positions are flying blind until earnings calls reveal demand collapsed. Smart money is paying for real-time shipment data from Korea, Taiwan, China because getting caught in a cycle turn costs more than the tracking fee.

60,621 просмотров • 1 месяц назад

At GTC 2024, Jensen said that GB200 NVL72 was 35x faster than Hopper. Nobody believed it and thought it was classic fake Jensen Math. When we tested the performance of it, it wasn't just 35x faster, it was over 50x times faster even against an strong Hopper baseline with all of the inference optimization composed together like MTP, Disagg prefill, wideEP, etc. View the nuanced results at InferenceX dot com.

At GTC 2024, Jensen said that GB200 NVL72 was 35x faster than Hopper. Nobody believed it and thought it was classic fake Jensen Math. When we tested the performance of it, it wasn't just 35x faster, it was over 50x times faster even against an strong Hopper baseline with all of the inference optimization composed together like MTP, Disagg prefill, wideEP, etc. View the nuanced results at InferenceX dot com.

60,803 просмотров • 1 месяц назад

296 hackers. 18,432 B200 GPU hours. $180K in compute credits. 60,000mg of caffeine. We kicked off GTC with Fluidstack for From Silicon to Scale — the most full-stack AI hackathon ever built. 48 hours. 1 DGX Spark signed by Jensen. Here's what happened.(1/7)🧵

296 hackers. 18,432 B200 GPU hours. $180K in compute credits. 60,000mg of caffeine. We kicked off GTC with Fluidstack for From Silicon to Scale — the most full-stack AI hackathon ever built. 48 hours. 1 DGX Spark signed by Jensen. Here's what happened.(1/7)🧵

69,870 просмотров • 2 месяцев назад

Jensen surfing a wave with the American flag is the most accurate metaphor for what's actually happening right now. A kid immigrates from the country of Taiwan, washes dishes at Denny's, and builds a $3 trillion company that every nation on earth is now begging for access to. He is the definition of the American dream. In order for America to strive, Earth needs to be build on American standards

Jensen surfing a wave with the American flag is the most accurate metaphor for what's actually happening right now. A kid immigrates from the country of Taiwan, washes dishes at Denny's, and builds a $3 trillion company that every nation on earth is now begging for access to. He is the definition of the American dream. In order for America to strive, Earth needs to be build on American standards

45,343 просмотров • 1 месяц назад

As we wrote in our TPUv7 article, TPUv7 will have competitive perf per TCO verus Blackwell in 2025 for large labs but TPUv8 will have worse perf per TCO verus Rubin in EOY 2026/2027. We added an subway surfer video as some people don't have the attention span to read an 10,000 word article. There will be 2 versions of TPU v8 but this will be different from the “P” (full) and “E” (lite) SKUs that featured in the then 4th and 5th generations. Rather, it is a dual track with one SKU co-designed with Broadcom (TPU 8AX ) and one SKU co-designed with MediaTek (TPU 8X). The strategy behind Google choosing to partner with MediaTek is to reduce the margin that they pay to their silicon design partners or specifically Broadcom. Broadcom charges Google for the whole System in Package which they stack a healthy margin on. This includes HBM. This is despite Google being largely responsible for both the front-end and back-end design of the compute elements of the chip, with Broadcom contributing various PHYs (most importantly Broadcom’s best in class SerDes) and controllers. Google wants to move to a model where their silicon partner only charges for the things they add value on and get closer to paying BOM cost for their silicon.

As we wrote in our TPUv7 article, TPUv7 will have competitive perf per TCO verus Blackwell in 2025 for large labs but TPUv8 will have worse perf per TCO verus Rubin in EOY 2026/2027. We added an subway surfer video as some people don't have the attention span to read an 10,000 word article. There will be 2 versions of TPU v8 but this will be different from the “P” (full) and “E” (lite) SKUs that featured in the then 4th and 5th generations. Rather, it is a dual track with one SKU co-designed with Broadcom (TPU 8AX ) and one SKU co-designed with MediaTek (TPU 8X). The strategy behind Google choosing to partner with MediaTek is to reduce the margin that they pay to their silicon design partners or specifically Broadcom. Broadcom charges Google for the whole System in Package which they stack a healthy margin on. This includes HBM. This is despite Google being largely responsible for both the front-end and back-end design of the compute elements of the chip, with Broadcom contributing various PHYs (most importantly Broadcom’s best in class SerDes) and controllers. Google wants to move to a model where their silicon partner only charges for the things they add value on and get closer to paying BOM cost for their silicon.

133,077 просмотров • 5 месяцев назад

Plot twist: the most valuable part of a 35-page semiconductor data report isn't the 200+ datasets or monthly charts. It's the one-page executive summary that tells you exactly which 3-5 trends actually matter and where to find them.

Plot twist: the most valuable part of a 35-page semiconductor data report isn't the 200+ datasets or monthly charts. It's the one-page executive summary that tells you exactly which 3-5 trends actually matter and where to find them.

34,783 просмотров • 1 месяц назад

"I don't need to change the world overnight. I'm gonna change the world over the next 50 years" - Jensen Huang, former Denny’s dishwasher

"I don't need to change the world overnight. I'm gonna change the world over the next 50 years" - Jensen Huang, former Denny’s dishwasher

179,735 просмотров • 8 месяцев назад

A common topic of discussion amongst SF Bay Area AI researchers is debating whether they should move to New York City. Something NYC has that the Bay Area does not have is the traditional Mexican frozen treat called nieves, run by Fidel Cortés Jr. and his family at a street-side stand. They hand-swirl freshly blended juice made from real fruits in large wooden barrels that are packed with ice and salt for around two hours.

A common topic of discussion amongst SF Bay Area AI researchers is debating whether they should move to New York City. Something NYC has that the Bay Area does not have is the traditional Mexican frozen treat called nieves, run by Fidel Cortés Jr. and his family at a street-side stand. They hand-swirl freshly blended juice made from real fruits in large wooden barrels that are packed with ice and salt for around two hours.

133,489 просмотров • 6 месяцев назад