Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Think DeepSpeed is only good for ZeRO and its usability, not training throughput? At #PyTorchCon North America 2026, Masahiro Tanaka will cover tensor, sequence, and expert parallelism in DeepSpeed and walk through benchmarks. His session, “DeepSpeed Is Not Just ZeRO: Tensor, Sequence, and Expert Parallelism with ZeRO-Level Usability,” will... show more

PyTorch

516,064 subscribers

15,698 views • 1 day ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

1 Comments

Patch1 day ago

@DeepSpeedAI The guidance on choosing and combining these strategies sounds useful.

Related Videos

At PyTorch Conference North America 2026, hear directly from and connect with the people working on PyTorch, vLLM, DeepSpeed, ray, Helion, and Safetensors, alongside experts working across the AI stack. At #PyTorchCon NA 2025, Richard Liaw (ray, Anyscale) spoke about the Ray team’s focus on building something “useful and critical to the AI infrastructure space” and about “how important Ray can be for the stack.” PyTorch conferences are the open source AI community’s town square, where what’s next gets decided. Register by September 4 (tomorrow) to save on your conference pass: #PyTorchCon

At PyTorch Conference North America 2026, hear directly from and connect with the people working on PyTorch, vLLM, DeepSpeed, ray, Helion, and Safetensors, alongside experts working across the AI stack. At #PyTorchCon NA 2025, Richard Liaw (ray, Anyscale) spoke about the Ray team’s focus on building something “useful and critical to the AI infrastructure space” and about “how important Ray can be for the stack.” PyTorch conferences are the open source AI community’s town square, where what’s next gets decided. Register by September 4 (tomorrow) to save on your conference pass: #PyTorchCon

PyTorch

16,342 views • 1 month ago

The open source AI community thrives on collaboration, choice, and open contribution. As Joseph Groenenboom from Red Hat highlights, empowering developers to freely contribute and build options is essential for growing the entire AI ecosystem. PyTorch Conference North America 2026 is where that community comes together in person. Join us to collaborate with open source pioneers, expand your deployment capabilities, and directly contribute to shaping the future of machine learning. Register for PyTorch Conference North America today: #PyTorchCon

The open source AI community thrives on collaboration, choice, and open contribution. As Joseph Groenenboom from Red Hat highlights, empowering developers to freely contribute and build options is essential for growing the entire AI ecosystem. PyTorch Conference North America 2026 is where that community comes together in person. Join us to collaborate with open source pioneers, expand your deployment capabilities, and directly contribute to shaping the future of machine learning. Register for PyTorch Conference North America today: #PyTorchCon

PyTorch

17,740 views • 26 days ago

At PyTorch Conference North America 2026, hear directly from and connect with the people working on PyTorch, vLLM, DeepSpeed, ray, Helion, and Safetensors, alongside experts working across the AI stack. Keynote speaker Simon Mo (vLLM, Inferact) says vLLM’s goal is to become “the easiest to use and most efficient inference engine.” At #PyTorchCon NA 2025, he spoke about the difficulty and cost of using large language models and continuing to push inference efficiency forward. PyTorch conferences are the open source AI community’s town square, where what’s next gets decided. Register by September 4 to save on your conference pass:

At PyTorch Conference North America 2026, hear directly from and connect with the people working on PyTorch, vLLM, DeepSpeed, ray, Helion, and Safetensors, alongside experts working across the AI stack. Keynote speaker Simon Mo (vLLM, Inferact) says vLLM’s goal is to become “the easiest to use and most efficient inference engine.” At #PyTorchCon NA 2025, he spoke about the difficulty and cost of using large language models and continuing to push inference efficiency forward. PyTorch conferences are the open source AI community’s town square, where what’s next gets decided. Register by September 4 to save on your conference pass:

PyTorch

22,334 views • 1 month ago

At #PyTorchCon North America 2026, Liz Li and Shekhar of AMD will discuss their work enabling scalable MXFP8 pretraining on MI355X GPUs. Liz and Shekhar will cover optimization of key MXFP8 operators in TorchAO and how those kernels were integrated with TorchTitan to build an end-to-end upstream PyTorch training stack for large language models. Their session, “Scaling MXFP8 Pretraining on 1K+ AMD Instinct MI355X: TorchAO Kernels and TorchTitan Training,” will also cover Triton and FlyDSL implementations and lessons from numerical validation, convergence, and performance tuning. Join us in San Jose on October 20-21. Register today:

At #PyTorchCon North America 2026, Liz Li and Shekhar of AMD will discuss their work enabling scalable MXFP8 pretraining on MI355X GPUs. Liz and Shekhar will cover optimization of key MXFP8 operators in TorchAO and how those kernels were integrated with TorchTitan to build an end-to-end upstream PyTorch training stack for large language models. Their session, “Scaling MXFP8 Pretraining on 1K+ AMD Instinct MI355X: TorchAO Kernels and TorchTitan Training,” will also cover Triton and FlyDSL implementations and lessons from numerical validation, convergence, and performance tuning. Join us in San Jose on October 20-21. Register today:

PyTorch

16,137 views • 14 days ago

Interested in optimizing Mixture of Experts LLM inference on ARM CPUs? In his talk at PyTorch Conference North America, Maajid Khan from Fujitsu Research India will explore efficient MoE LLM inference using vLLM and OpenVINO, sharing practical strategies for running these models effectively on ARM architectures. Register and join us in San Jose, October 20-21: #PyTorchCon

Interested in optimizing Mixture of Experts LLM inference on ARM CPUs? In his talk at PyTorch Conference North America, Maajid Khan from Fujitsu Research India will explore efficient MoE LLM inference using vLLM and OpenVINO, sharing practical strategies for running these models effectively on ARM architectures. Register and join us in San Jose, October 20-21: #PyTorchCon

PyTorch

16,448 views • 13 days ago

"It's really great to feel the energy and see all the collaboration that's happening, lots of conversations. We have people here who are just starting to learn about PyTorch, and we have people who are maintainers. It's super nice to see that energy and see how together we're going to be able to solve some of these very difficult problems for humanity." Ankit Patel, NVIDIA, at PyTorch Conference Europe 2026 Register today to join the open source AI community at PyTorch Conference North America in San Jose, October 20–21: #PyTorchCon

"It's really great to feel the energy and see all the collaboration that's happening, lots of conversations. We have people here who are just starting to learn about PyTorch, and we have people who are maintainers. It's super nice to see that energy and see how together we're going to be able to solve some of these very difficult problems for humanity." Ankit Patel, NVIDIA, at PyTorch Conference Europe 2026 Register today to join the open source AI community at PyTorch Conference North America in San Jose, October 20–21: #PyTorchCon

PyTorch

17,000 views • 25 days ago

Build a neural network from scratch, train it, and optimize it for real-world AI infrastructure during our PyTorch Associate Training the day before PyTorch Conference North America on October 19th. Your instructor, Faradawn Yang of NVIDIA AI explains how this training will give you hands on experience with the latest AI accelerators & prepare you for the PyTorch Certified Associate (PTCA) certification exam. #PyTorchCon

Build a neural network from scratch, train it, and optimize it for real-world AI infrastructure during our PyTorch Associate Training the day before PyTorch Conference North America on October 19th. Your instructor, Faradawn Yang of NVIDIA AI explains how this training will give you hands on experience with the latest AI accelerators & prepare you for the PyTorch Certified Associate (PTCA) certification exam. #PyTorchCon

PyTorch

17,476 views • 10 days ago

From torch.profiler to Hardware Cycles: A Practical Profiling Playbook AWS Trainium: designed from the ground up for next generation AI training and inference is now a standard PyTorch device. When you need to optimize, the tools are built in. Esha Lakhotia and Pinak Panigrahi of Amazon Web Services will walk you from torch.profiler capture to cycle-level hardware debugging, with AI-assisted analysis that suggests fixes at PyTorch Conference North America. 📅 Oct 21 | 12:20 PM | LL21ABC Join us in San Jose, October 20-21: #PyTorchCon

From torch.profiler to Hardware Cycles: A Practical Profiling Playbook AWS Trainium: designed from the ground up for next generation AI training and inference is now a standard PyTorch device. When you need to optimize, the tools are built in. Esha Lakhotia and Pinak Panigrahi of Amazon Web Services will walk you from torch.profiler capture to cycle-level hardware debugging, with AI-assisted analysis that suggests fixes at PyTorch Conference North America. 📅 Oct 21 | 12:20 PM | LL21ABC Join us in San Jose, October 20-21: #PyTorchCon

PyTorch

14,492 views • 2 days ago

New chips are shipping faster than ever before, but the ecosystem is being held back by having to rewrite and then re-debug the same code over & over again. Chris Lattner CEO and Co-founder at Modular and EVP of Advanced AI Software & Platforms at Qualcomm, will deliver a keynote at PyTorch Conference North America about an open software platform for heterogeneous compute powered by Mojo and MAX. PyTorch has always been the place where the best models come together, and now there's a way to get those models onto all kinds of hardware. If you're interested in Al and compute, join us at the PyTorch Conference in San Jose, CA. Register now: #PyTorchCon

New chips are shipping faster than ever before, but the ecosystem is being held back by having to rewrite and then re-debug the same code over & over again. Chris Lattner CEO and Co-founder at Modular and EVP of Advanced AI Software & Platforms at Qualcomm, will deliver a keynote at PyTorch Conference North America about an open software platform for heterogeneous compute powered by Mojo and MAX. PyTorch has always been the place where the best models come together, and now there's a way to get those models onto all kinds of hardware. If you're interested in Al and compute, join us at the PyTorch Conference in San Jose, CA. Register now: #PyTorchCon

PyTorch

20,090 views • 23 days ago

PyTorch Conference is where the open source Al community gathers to talk about what we are all building together. Priya Nagpurkar, VP of AI Native Systems at IBM Research will give a keynote at #PyTorchCon North America on IBM & Red Hat's contributions to the PyTorch ecosystem, including exciting developments on infrastructure for out-of-tree accelerators, enablement of IBM's Spyre Accelerator, and work with Ilm-d for distributed inference. Come to the talk, meet the community, and hear what's next in open source Al. Join us in San Jose, October 20-21:

PyTorch Conference is where the open source Al community gathers to talk about what we are all building together. Priya Nagpurkar, VP of AI Native Systems at IBM Research will give a keynote at #PyTorchCon North America on IBM & Red Hat's contributions to the PyTorch ecosystem, including exciting developments on infrastructure for out-of-tree accelerators, enablement of IBM's Spyre Accelerator, and work with Ilm-d for distributed inference. Come to the talk, meet the community, and hear what's next in open source Al. Join us in San Jose, October 20-21:

PyTorch

16,019 views • 3 days ago

Alex Karp says AI companies don't understand the world is zero-sum. "[Silicon Valley companies] do “think it’s zero-sum: they think it’s zero-sum versus each other.” "They absolutely are fighting. It's my own rhetoric, but I happen to think it's true: it's going to be chips ontology, and I suspect, in the end, one-and-a-half providers." "They pretend they don't think it's zero-sum, but then ask them how they feel about their competitor... these people are fighting very, very hard for the dominant position." "What they don't understand is that in the world, it's zero-sum. But I actually think the primary issue in the valley is it's going to be zero-sum vis-à-vis you." "When politicians figure out this is the one winning issue, it's going to be zero-sum. It's going to be your money and your company being zero-sum nationalized. So that's the part they don't understand." "It seems to be very hard in this culture... to explain to people that we're in a competitive environment, because things here are so good. They don't understand how different it will be; and how much we will change; and how much the pressure will be on America, not just legally, but militarily and culturally, if we fall behind." Palantir Katherine Boyle

Alex Karp says AI companies don't understand the world is zero-sum. "[Silicon Valley companies] do “think it’s zero-sum: they think it’s zero-sum versus each other.” "They absolutely are fighting. It's my own rhetoric, but I happen to think it's true: it's going to be chips ontology, and I suspect, in the end, one-and-a-half providers." "They pretend they don't think it's zero-sum, but then ask them how they feel about their competitor... these people are fighting very, very hard for the dominant position." "What they don't understand is that in the world, it's zero-sum. But I actually think the primary issue in the valley is it's going to be zero-sum vis-à-vis you." "When politicians figure out this is the one winning issue, it's going to be zero-sum. It's going to be your money and your company being zero-sum nationalized. So that's the part they don't understand." "It seems to be very hard in this culture... to explain to people that we're in a competitive environment, because things here are so good. They don't understand how different it will be; and how much we will change; and how much the pressure will be on America, not just legally, but militarily and culturally, if we fall behind." Palantir Katherine Boyle

a16z

124,595 views • 7 months ago

Welcome to CPAC 2026. Thousands gathered. Not for ideas. Not for solutions. Just spectacle. Costumes. Merch. Distraction. While real problems pile up, this is what leadership looks like. Sequins, slogans, and zero substance. Follow Save America Movement for coverage that cuts through the performance.

Welcome to CPAC 2026. Thousands gathered. Not for ideas. Not for solutions. Just spectacle. Costumes. Merch. Distraction. While real problems pile up, this is what leadership looks like. Sequins, slogans, and zero substance. Follow Save America Movement for coverage that cuts through the performance.

Save America Movement

115,540 views • 6 months ago

Want to hear more about new vLLM and SGLang engines built on a new PyTorch-native TPU backend, TorchTPU? Find Qi Zhou of Google, Colin Taylor and Angela Yi of Meta at PyTorch Conference North America in San Jose where they will discuss how they focused on making the existing SGLang & vLLM infrastructure work natively on TPU, preserving the schedulers, batching systems, OpenAI-compatible APIs, and torch.compile workflows already familiar to GPU users. Because the serving engines remain upstream, new models and features can run on TPU without requiring separate TPU-specific implementations. Learn more about how it works across the stacks & what they learned running this in production, including compile time, device placement under tracing, KV cache layout, and cross-chip collectives. Register for #PyTorchCon NA now:

Want to hear more about new vLLM and SGLang engines built on a new PyTorch-native TPU backend, TorchTPU? Find Qi Zhou of Google, Colin Taylor and Angela Yi of Meta at PyTorch Conference North America in San Jose where they will discuss how they focused on making the existing SGLang & vLLM infrastructure work natively on TPU, preserving the schedulers, batching systems, OpenAI-compatible APIs, and torch.compile workflows already familiar to GPU users. Because the serving engines remain upstream, new models and features can run on TPU without requiring separate TPU-specific implementations. Learn more about how it works across the stacks & what they learned running this in production, including compile time, device placement under tracing, KV cache layout, and cross-chip collectives. Register for #PyTorchCon NA now:

PyTorch

16,759 views • 7 days ago

What You Cannot Profile, You Cannot Optimize: Learning to Read PyTorch Traces In his talk at PyTorch Conference North America 2026, Suvaditya Mukherjee from Hugging Face will share practical guidance on how to start using the PyTorch Profiler on your models to extract maximum performance efficiency. Whether you are aiming to identify performance bottlenecks or optimize model execution, this talk will help you get the most out of your hardware. Join the open source AI community in San Jose, October 20-21: #PyTorchCon Suvaditya Mukherjee

PyTorch

16,264 views • 8 days ago

Mike Huckabee doubles down on his claims that Christianity would not exist without Judaism and that Western civilization would not exist without Judeo-Christian values. Huckabee says this is why Israel and America will be tied together for eternity. Zero Jewish people signed the Declaration of Independence. "Without that, there would not be an America. It is as simple as that."

Mike Huckabee doubles down on his claims that Christianity would not exist without Judaism and that Western civilization would not exist without Judeo-Christian values. Huckabee says this is why Israel and America will be tied together for eternity. Zero Jewish people signed the Declaration of Independence. "Without that, there would not be an America. It is as simple as that."

Shadow of Ezra

121,299 views • 3 months ago

Liang Zhang (Huawei) delivers “Ascend & PyTorch: Pioneer New AI Open Ecosystem” at PyTorch Conference China 2026. Drawing on eight years of open source practice, Liang Zhang looks at how Ascend’s collaboration with the PyTorch community is evolving from hardware adaptation to joint research on technical standards. The keynote also covers next-generation AI hardware-software codesign, interoperability between Chinese and global AI technology stacks, and building an open and sustainable AI industrial ecosystem. Here's Liang Zhang, introduced by Mark Collier 柯理怀: #PyTorchCon

Liang Zhang (Huawei) delivers “Ascend & PyTorch: Pioneer New AI Open Ecosystem” at PyTorch Conference China 2026. Drawing on eight years of open source practice, Liang Zhang looks at how Ascend’s collaboration with the PyTorch community is evolving from hardware adaptation to joint research on technical standards. The keynote also covers next-generation AI hardware-software codesign, interoperability between Chinese and global AI technology stacks, and building an open and sustainable AI industrial ecosystem. Here's Liang Zhang, introduced by Mark Collier 柯理怀: #PyTorchCon

PyTorch

16,367 views • 1 month ago

🇺🇸🇮🇷 The clock is ticking on U.S. oil reserves, and Iran doesn't need to fire a single shot. Geopolitics Expert Brandon Weichert says Tehran's entire strategy is to sit back and watch America speed toward zero. "Time is an advantage for the Iranians in this war. It is not an advantage for the United States." Push a country to the edge, and it stops thinking straight. That's when it loses. Brandon Weichert

🇺🇸🇮🇷 The clock is ticking on U.S. oil reserves, and Iran doesn't need to fire a single shot. Geopolitics Expert Brandon Weichert says Tehran's entire strategy is to sit back and watch America speed toward zero. "Time is an advantage for the Iranians in this war. It is not an advantage for the United States." Push a country to the edge, and it stops thinking straight. That's when it loses. Brandon Weichert

Mario Nawfal

67,898 views • 3 months ago

PATHETIC: ABC, CBS, and NBC gave these stories zero seconds. Not one. Illegal aliens accused of stalking and killing a woman in Sacramento. Crushing a nine-month-old in New Jersey. Strangling a woman in Massachusetts. Stabbing a coworker in Fairfax. Dealing fentanyl near a Loudoun County school. Sanctuary after sanctuary. Silence after silence. Trump secured the border. Zero crossings became real. Now ICE is cleaning up the monsters Biden’s chaos left behind, and the networks still won’t show the bodies. They will cover a protest in a minute. They will not cover a dead American if it makes open borders look guilty. That is not journalism. That is a cover-up with a teleprompter.

PATHETIC: ABC, CBS, and NBC gave these stories zero seconds. Not one. Illegal aliens accused of stalking and killing a woman in Sacramento. Crushing a nine-month-old in New Jersey. Strangling a woman in Massachusetts. Stabbing a coworker in Fairfax. Dealing fentanyl near a Loudoun County school. Sanctuary after sanctuary. Silence after silence. Trump secured the border. Zero crossings became real. Now ICE is cleaning up the monsters Biden’s chaos left behind, and the networks still won’t show the bodies. They will cover a protest in a minute. They will not cover a dead American if it makes open borders look guilty. That is not journalism. That is a cover-up with a teleprompter.

Gunther Eagleman™

56,573 views • 19 days ago

Dan Hooker says BSD will realize fast that he’s facing a man with zero concern for his health and safety 😳🥶 "It'll be funny when he gets in there and I'll just start slamming into him, and he'll be like, 'Oh wow, he's not concerned about his health and safety at all.'" (via MAIN EVENT )

Dan Hooker says BSD will realize fast that he’s facing a man with zero concern for his health and safety 😳🥶 "It'll be funny when he gets in there and I'll just start slamming into him, and he'll be like, 'Oh wow, he's not concerned about his health and safety at all.'" (via MAIN EVENT )

Red Corner MMA

342,700 views • 10 months ago