正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

MI355 disaggregated serving is competitive to B200 disaggregated serving for FP8 but when composability all the optimizations that frontier labs use together like wide expert parallelism+disagg+FP4+kvcache offloading AMD is still jestermaxxing & lackluster. King Anush Elangovan needs to focus on composability of inference optimizations for ROCm stack in order... show more

SemiAnalysis

65,058 subscribers

22,416 次观看 • 4 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Something NVIDIA & Google do better than anyone else is software-hardware-system co-design, and not just optimizing hardware for current model architectures, but predicting future ones. Back in early 2022, when NVIDIA started the design process for NVL72, MoE (Mixture of Experts) models were not yet the standard, and dense models were still dominant for frontier models. However, NVIDIA's strong software-hardware co-design culture enabled them to make a calculated bet that MoEs were the future, and they built NVL72 specifically for best MoE performance per TCO (Total Cost of Ownership). Furthermore, back in 2022, disaggregated prefill and wide expert parallelism (wideEP) MoE inference optimizations hadn't been invented yet, but it turns out that these MoE inference optimizations work best on large-scale systems like NVL72. While most other AI chip companies' in-house AI labs focus on training small 5B models that mainly use data parallelism, NVIDIA and Google's in-house AI labs continuously push the boundaries of model architecture and training recipes, such as NVFP4 training. Just like Super Idol & IShowSpeed, there must be a strong partnership between software engineers and hardware engineers to deliver the best systems that maximize performance per TCO.

Something NVIDIA & Google do better than anyone else is software-hardware-system co-design, and not just optimizing hardware for current model architectures, but predicting future ones. Back in early 2022, when NVIDIA started the design process for NVL72, MoE (Mixture of Experts) models were not yet the standard, and dense models were still dominant for frontier models. However, NVIDIA's strong software-hardware co-design culture enabled them to make a calculated bet that MoEs were the future, and they built NVL72 specifically for best MoE performance per TCO (Total Cost of Ownership). Furthermore, back in 2022, disaggregated prefill and wide expert parallelism (wideEP) MoE inference optimizations hadn't been invented yet, but it turns out that these MoE inference optimizations work best on large-scale systems like NVL72. While most other AI chip companies' in-house AI labs focus on training small 5B models that mainly use data parallelism, NVIDIA and Google's in-house AI labs continuously push the boundaries of model architecture and training recipes, such as NVFP4 training. Just like Super Idol & IShowSpeed, there must be a strong partnership between software engineers and hardware engineers to deliver the best systems that maximize performance per TCO.

SemiAnalysis

51,021 次观看 • 8 个月前

AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's Anush Elangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's Anush Elangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

SemiAnalysis

53,783 次观看 • 8 个月前

MRC is already deployed across all of OpenAI’s largest supercomputers that we use to train frontier models, including our site with Oracle Cloud Infrastructure (OCI) in Abilene, Texas, and in Microsoft’s Fairwater supercomputers. MRC is now available through the Open Compute Project for the entire industry to use and build on.

MRC is already deployed across all of OpenAI’s largest supercomputers that we use to train frontier models, including our site with Oracle Cloud Infrastructure (OCI) in Abilene, Texas, and in Microsoft’s Fairwater supercomputers. MRC is now available through the Open Compute Project for the entire industry to use and build on.

OpenAI

145,781 次观看 • 2 个月前

Establishing 'Pressing triggers' is about ensuring the team are on the same wavelength of when & how to act when they do not have the ball, in order to find the best opportunity to be aggressive in regaining it. Re-watching the Brighton game, in reference to the past 11 months, it's apparent that this is still an issue for Amorim's united in preventing them from being easy to attack.

Establishing 'Pressing triggers' is about ensuring the team are on the same wavelength of when & how to act when they do not have the ball, in order to find the best opportunity to be aggressive in regaining it. Re-watching the Brighton game, in reference to the past 11 months, it's apparent that this is still an issue for Amorim's united in preventing them from being easy to attack.

.

29,398 次观看 • 9 个月前

$🍩Day is coming to Chicago on 11/19! We’re serving up BIG things with Amazon. Stay tuned for the pop up address & info! Since the best things in life are free (like Grubhub+ for Amazon Prime members), we’re giving away a $500 Grubhub gift card ✨ Want to enter for your chance to win? - Tell us your favorite Grubhub+ order - Tag a friend - Use #ZeroDollarDaySweepstakes Prime members: activate your free Grubhub+ now at

$🍩Day is coming to Chicago on 11/19! We’re serving up BIG things with Amazon. Stay tuned for the pop up address & info! Since the best things in life are free (like Grubhub+ for Amazon Prime members), we’re giving away a $500 Grubhub gift card ✨ Want to enter for your chance to win? - Tell us your favorite Grubhub+ order - Tag a friend - Use #ZeroDollarDaySweepstakes Prime members: activate your free Grubhub+ now at

Grubhub

25,318 次观看 • 1 年前

Don’t be like this player. So many players still don’t use boost efficiently, even though boost management is the most important skill in the game. A good guideline is to save boost for when you’re making a play on the ball and use your movement to get to the ball. Here is a player who uses all of his boost to chase the ball, and when he ends up low on boost, he keeps chasing anyway. This leads to him bumping his teammate and giving up an easy goal.

Don’t be like this player. So many players still don’t use boost efficiently, even though boost management is the most important skill in the game. A good guideline is to save boost for when you’re making a play on the ball and use your movement to get to the ball. Here is a player who uses all of his boost to chase the ball, and when he ends up low on boost, he keeps chasing anyway. This leads to him bumping his teammate and giving up an easy goal.

COACH KINSEH

20,370 次观看 • 7 个月前

When some people in England recoil at the thought of bombing the IRGC and the mad Mullahs in Iran I would just like to remind them.... When they call for the death of England...... I say give them death. That is all, thanks for your attention on this matter.

When some people in England recoil at the thought of bombing the IRGC and the mad Mullahs in Iran I would just like to remind them.... When they call for the death of England...... I say give them death. That is all, thanks for your attention on this matter.

Tommy Robinson 🇬🇧

212,318 次观看 • 5 个月前

To westerners, this video looks like a normal, everyday occurrence in town. But in occupied Iran, this is what civil disobedience looks like: uncovered hair, young people dancing, and western music. All things that are banned by Sharia Law. Don't be fooled by the pro-regime propagandists who will use this video to claim that "Iran is free" and "No need to overthrow the Islamic Republic". Iran is still occupied by the terrorist Islamic Republic. Iranians are being executed every day in the name of Islam for the crime of speaking out and wanting to live in a free and secular society. This is just a glimpse of the slow crumble of Islamic oppression on a society that yearns for freedom. Expect to see more signs of civil disobedience as the regime continues to crumble and collapse. And expect to see more disinformation coming out, claiming that Iran is "already free", by those who are trying desperately to keep the Islamic Republic occupying Iran in power.

To westerners, this video looks like a normal, everyday occurrence in town. But in occupied Iran, this is what civil disobedience looks like: uncovered hair, young people dancing, and western music. All things that are banned by Sharia Law. Don't be fooled by the pro-regime propagandists who will use this video to claim that "Iran is free" and "No need to overthrow the Islamic Republic". Iran is still occupied by the terrorist Islamic Republic. Iranians are being executed every day in the name of Islam for the crime of speaking out and wanting to live in a free and secular society. This is just a glimpse of the slow crumble of Islamic oppression on a society that yearns for freedom. Expect to see more signs of civil disobedience as the regime continues to crumble and collapse. And expect to see more disinformation coming out, claiming that Iran is "already free", by those who are trying desperately to keep the Islamic Republic occupying Iran in power.

Goldie Ghamari | گلسا قمری

28,261 次观看 • 9 个月前

He's not wrong lol. HR has a ceiling & is meaningless in front of characters who can use CE to obtain higher stats.. But that's only when we're talking about people like Yuta & beyond.. For most Maki's HR is insane. It's the equivalent of having that level of CE reinforcement on at all times, effortlessly. Like Gojo's barrier.. #JujutsuKaisen #JJK

He's not wrong lol. HR has a ceiling & is meaningless in front of characters who can use CE to obtain higher stats.. But that's only when we're talking about people like Yuta & beyond.. For most Maki's HR is insane. It's the equivalent of having that level of CE reinforcement on at all times, effortlessly. Like Gojo's barrier.. #JujutsuKaisen #JJK

Shadz

150,737 次观看 • 6 个月前

The state, in its infinite wisdom, has decided that the best way to deal with a woman who speaks for the unheard is to ensure she is unheard too. Mahrang Baloch, a leader too genuine for the carefully curated democracy around her, now sits behind bars—not for any crime, but for the far greater offense of making the voiceless believe they could have a voice. Of course, they call it “maintaining order.” Because nothing disturbs order quite like a woman who refuses to sit down, who dares to ask why the disappeared are still missing, why justice is always delayed, and why the only thing that moves swiftly in this country is the hand that silences dissent. #MahrangBaloch #Women #Quetta

The state, in its infinite wisdom, has decided that the best way to deal with a woman who speaks for the unheard is to ensure she is unheard too. Mahrang Baloch, a leader too genuine for the carefully curated democracy around her, now sits behind bars—not for any crime, but for the far greater offense of making the voiceless believe they could have a voice. Of course, they call it “maintaining order.” Because nothing disturbs order quite like a woman who refuses to sit down, who dares to ask why the disappeared are still missing, why justice is always delayed, and why the only thing that moves swiftly in this country is the hand that silences dissent. #MahrangBaloch #Women #Quetta

Hazaran Baloch

16,676 次观看 • 1 年前

President Trump made putting an end to child trafficking a focus point of his first administration and that hasn’t changed. Anyone who has supported Trump from the beginning knows that to be a fact. When he says things like “I don’t understand why the Jeffrey Epstein case would be of interest to anybody” … he does not mean that, and if you believe that, then you have simply forgotten how Trump plays chess in order to make his opponents do what he wants. And it’s working phenomenally. All of the people on the right who refused to talk about Epstein when all of the Anons were, are now screaming for the Epstein files. All of the people on the left who called us conspiracy theorists, including the pedophiles themselves, are now screaming for the Epstein files. “When he says things like “Really only pretty bad people, including fake news want to keep something like that going” … it is more of the same tactic. It sounds so ridiculous that Howard Lutnick burst out laughing! This has always been about the children, and it still is. Protect your mind and control your emotions.

President Trump made putting an end to child trafficking a focus point of his first administration and that hasn’t changed. Anyone who has supported Trump from the beginning knows that to be a fact. When he says things like “I don’t understand why the Jeffrey Epstein case would be of interest to anybody” … he does not mean that, and if you believe that, then you have simply forgotten how Trump plays chess in order to make his opponents do what he wants. And it’s working phenomenally. All of the people on the right who refused to talk about Epstein when all of the Anons were, are now screaming for the Epstein files. All of the people on the left who called us conspiracy theorists, including the pedophiles themselves, are now screaming for the Epstein files. “When he says things like “Really only pretty bad people, including fake news want to keep something like that going” … it is more of the same tactic. It sounds so ridiculous that Howard Lutnick burst out laughing! This has always been about the children, and it still is. Protect your mind and control your emotions.

RealAF Patriot

19,722 次观看 • 1 年前

And every single thing is recorded in 2021——2025 National Defense Authorization Acts… BEFORE anything was on “News” which means PRE—PLANNED… Outside of the fact the Law more than clearly shows we are still in a Partial Continuity of Government… Which means Congress are under a Military Government and Occupation, all theatre… For those who just cannot grasp that — it still accomplishes the mission… Expose Hypocrisy. When you look at all those who call PDJT a Dictator, Nazi, King, and Threat… Majority of those voted Yea on ALL the terminology in the DEFENSE BUDGETS for the Military that allowed it… Which is why 99.9999999% of “influencers” look unintelligent but should be charged with sedition for ignorantly misleading Americans with surface level nonsense… When there is a CLEAR answer via Law that is designed BIPARTISAN for ALL AMERICANS. Anyone who keeps the partisan up is missing the | 45 ——— 47 | which is Strategic and Special Operations due to the vast ignorance. 🫡♠️🇺🇸

And every single thing is recorded in 2021——2025 National Defense Authorization Acts… BEFORE anything was on “News” which means PRE—PLANNED… Outside of the fact the Law more than clearly shows we are still in a Partial Continuity of Government… Which means Congress are under a Military Government and Occupation, all theatre… For those who just cannot grasp that — it still accomplishes the mission… Expose Hypocrisy. When you look at all those who call PDJT a Dictator, Nazi, King, and Threat… Majority of those voted Yea on ALL the terminology in the DEFENSE BUDGETS for the Military that allowed it… Which is why 99.9999999% of “influencers” look unintelligent but should be charged with sedition for ignorantly misleading Americans with surface level nonsense… When there is a CLEAR answer via Law that is designed BIPARTISAN for ALL AMERICANS. Anyone who keeps the partisan up is missing the | 45 ——— 47 | which is Strategic and Special Operations due to the vast ignorance. 🫡♠️🇺🇸

Derek Johnson

11,499 次观看 • 4 个月前

Irish people, pay attention to what's happening in Minneapolis because a similar situation is developing in Ireland. 1,500 migrants are planning to descend on government buildings next week to demand amnesty and an end to all deportations. They're promising "no justice, no peace." Did you know a left-wing terrorist named Stephen Bedford is currently serving a prison sentence for driving his car through a crowd of Irish people who were calling for deportations? Similar to the US, migrants, NGOs and the left are working together to bring chaos to our streets.

Irish people, pay attention to what's happening in Minneapolis because a similar situation is developing in Ireland. 1,500 migrants are planning to descend on government buildings next week to demand amnesty and an end to all deportations. They're promising "no justice, no peace." Did you know a left-wing terrorist named Stephen Bedford is currently serving a prison sentence for driving his car through a crowd of Irish people who were calling for deportations? Similar to the US, migrants, NGOs and the left are working together to bring chaos to our streets.

MichaeloKeeffe

126,776 次观看 • 6 个月前

In Guangzhou, a foreign woman living with lower-body paralysis tried a Chinese-made exoskeleton for the first time — and took her first steps on her own. The joy on her face was incredible. Her family was moved to tears. The Chinese team rushed over with big smiles and warm hugs. This is the beautiful side of technology: not a tool for dominance or conflict, but a bridge to help people walk again, live again, and smile again. When innovation focuses on serving humanity, we all move forward together. A more compassionate future is possible.

In Guangzhou, a foreign woman living with lower-body paralysis tried a Chinese-made exoskeleton for the first time — and took her first steps on her own. The joy on her face was incredible. Her family was moved to tears. The Chinese team rushed over with big smiles and warm hugs. This is the beautiful side of technology: not a tool for dominance or conflict, but a bridge to help people walk again, live again, and smile again. When innovation focuses on serving humanity, we all move forward together. A more compassionate future is possible.

Eivor

104,170 次观看 • 3 个月前

A year ago, I called Nvidia the literal bargaining chip in US-China trade. Today the thesis is no longer mine alone. Anthropic just published "2028: Two scenarios for global AI leadership" arguing compute is the entire game. Close the smuggling loopholes, kill distillation attacks, lock in a 12-24 month US lead. Same day, Reuters: US Commerce approved H200 sales to Alibaba, Tencent, ByteDance, Up to 75,000 chips each. Lenovo and Foxconn cleared as distributors. Jensen is in China this week trying to convert paper approvals into actual deliveries. The math: China was 13% of Nvidia revenue ($17B in FY25). Jensen at GTC: "$50B opportunity in 2025 alone, growing 50% annually." To CNBC in October: "a couple hundred billion by the end of the decade." That's the prize. But the prize is not just the revenue. It's the CUDA lock-in. On Dwarkesh in April, Jensen said the moat is not the silicon. The moat is the install base. Every cloud. Every robot. Every developer trained on CUDA. If China spins up on Huawei Ascend, that is a parallel stack that compounds against Nvidia forever. Concede the second largest compute market, you concede the ecosystem. This is why Jensen got visibly agitated when Dwarkesh pushed. "You are not talking to somebody who woke up a loser." It is also why Beijing is slow-walking the H200 orders. They understand the same thing in reverse. Every CUDA developer is a Huawei customer they will never get back. The bargaining chip is leverage in both directions. Anthropic's policy paper today is the US position. Jensen's posture is the corporate position. Beijing's go-slow is the Chinese position. All three agree on one thing: whoever owns the compute, owns the future. Memory Wars. Co-Design. The Reasoning Tax. All downstream of this. Compute is the unit of national power in the AI era. $NVDA

A year ago, I called Nvidia the literal bargaining chip in US-China trade. Today the thesis is no longer mine alone. Anthropic just published "2028: Two scenarios for global AI leadership" arguing compute is the entire game. Close the smuggling loopholes, kill distillation attacks, lock in a 12-24 month US lead. Same day, Reuters: US Commerce approved H200 sales to Alibaba, Tencent, ByteDance, Up to 75,000 chips each. Lenovo and Foxconn cleared as distributors. Jensen is in China this week trying to convert paper approvals into actual deliveries. The math: China was 13% of Nvidia revenue ($17B in FY25). Jensen at GTC: "$50B opportunity in 2025 alone, growing 50% annually." To CNBC in October: "a couple hundred billion by the end of the decade." That's the prize. But the prize is not just the revenue. It's the CUDA lock-in. On Dwarkesh in April, Jensen said the moat is not the silicon. The moat is the install base. Every cloud. Every robot. Every developer trained on CUDA. If China spins up on Huawei Ascend, that is a parallel stack that compounds against Nvidia forever. Concede the second largest compute market, you concede the ecosystem. This is why Jensen got visibly agitated when Dwarkesh pushed. "You are not talking to somebody who woke up a loser." It is also why Beijing is slow-walking the H200 orders. They understand the same thing in reverse. Every CUDA developer is a Huawei customer they will never get back. The bargaining chip is leverage in both directions. Anthropic's policy paper today is the US position. Jensen's posture is the corporate position. Beijing's go-slow is the Chinese position. All three agree on one thing: whoever owns the compute, owns the future. Memory Wars. Co-Design. The Reasoning Tax. All downstream of this. Compute is the unit of national power in the AI era. $NVDA

Ben Pouladian

14,452 次观看 • 2 个月前

Lady Gaga says you have committed violence by objecting to her celebration of Dylan Mulvaney on International Women’s Day. “It's appalling to me that a post about National Women's Day by Dylan Mulvaney and me would be met with such vitriol and hatred. When I see a newspaper reporting on hatred but calling it ‘backlash’ I feel it is important to clarify that hatred is hatred, and this kind of hatred is violence. “Backlash would imply that people who love or respect Dylan and me didn't like something we did. This is not backlash. This is hatred. “But it is not surprising given the immense work that it's obvious we still have to do as a society to make room for transgender lives to be cherished and upheld by all of us. “I feel very protective in this moment, not only of Dylan, but of the trans community who continues to lead the way with their endless grace and inspiration in the face of constant degradation, intolerance, and physical, verbal, and mental violence. I certainly do not speak for this community, but I have something to say. I hope all women will come together to honor us ALL for International Women's Day, and may we do that always until THE DAY that all women are celebrated equally. That all people are celebrated equally. A day where people of all gender identities are celebrated on whichever holiday speaks to them. Because people of all gender identities and races deserve peace and dignity. “May we all come together and be loving, accepting, warm, welcoming. May we all stand and honor the complexity and challenge of trans life-that we do not know, but can seek to understand and have compassion for. I love people too much to allow hatred to be referred to as ‘backlash.’ People deserve better.”

Lady Gaga says you have committed violence by objecting to her celebration of Dylan Mulvaney on International Women’s Day. “It's appalling to me that a post about National Women's Day by Dylan Mulvaney and me would be met with such vitriol and hatred. When I see a newspaper reporting on hatred but calling it ‘backlash’ I feel it is important to clarify that hatred is hatred, and this kind of hatred is violence. “Backlash would imply that people who love or respect Dylan and me didn't like something we did. This is not backlash. This is hatred. “But it is not surprising given the immense work that it's obvious we still have to do as a society to make room for transgender lives to be cherished and upheld by all of us. “I feel very protective in this moment, not only of Dylan, but of the trans community who continues to lead the way with their endless grace and inspiration in the face of constant degradation, intolerance, and physical, verbal, and mental violence. I certainly do not speak for this community, but I have something to say. I hope all women will come together to honor us ALL for International Women's Day, and may we do that always until THE DAY that all women are celebrated equally. That all people are celebrated equally. A day where people of all gender identities are celebrated on whichever holiday speaks to them. Because people of all gender identities and races deserve peace and dignity. “May we all come together and be loving, accepting, warm, welcoming. May we all stand and honor the complexity and challenge of trans life-that we do not know, but can seek to understand and have compassion for. I love people too much to allow hatred to be referred to as ‘backlash.’ People deserve better.”

Billboard Chris 🌎

1,498,971 次观看 • 2 年前

I'm sure you've all noticed the 'AI is slowing down' news stories every few weeks for multiple years now - so I've pulled a tracker together to see who and when wrote these stories. There is quite a range, some are just outright wrong, others point to a reasonable limitation at the time but missing the bigger arc of progress. All of these stories were appearing as we were getting reasoning models, open source models, increasing competition from more players and skyrocketing revenue for the labs. Link to the tracker in the comments

I'm sure you've all noticed the 'AI is slowing down' news stories every few weeks for multiple years now - so I've pulled a tracker together to see who and when wrote these stories. There is quite a range, some are just outright wrong, others point to a reasonable limitation at the time but missing the bigger arc of progress. All of these stories were appearing as we were getting reasoning models, open source models, increasing competition from more players and skyrocketing revenue for the labs. Link to the tracker in the comments

Peter Gostev

37,443 次观看 • 11 个月前

Doug Ford blamed patients for ER wait times yesterday. At this point, it’s not surprising that he won’t take accountability for his failures on healthcare. He’s been running from his record all campaign. But it is shocking, even for him, to suggest that the problem with our ERs, is that people have the audacity to rely on them when they are sick. What does another 4 years of leadership like this look like? The answer is worse than we can probably imagine. We can’t afford that. Vote Liberal to stop him on February 27th.

Doug Ford blamed patients for ER wait times yesterday. At this point, it’s not surprising that he won’t take accountability for his failures on healthcare. He’s been running from his record all campaign. But it is shocking, even for him, to suggest that the problem with our ERs, is that people have the audacity to rely on them when they are sick. What does another 4 years of leadership like this look like? The answer is worse than we can probably imagine. We can’t afford that. Vote Liberal to stop him on February 27th.

Bonnie Crombie 🇨🇦

56,295 次观看 • 1 年前