Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Learn how teamed up with Modular and Oracle Cloud to unlock hardware optionality across Nvidia and AMD GPUs for their groundbreaking TTS models, cutting total cost of model ownership by 70%.

Modular

24,345 subscribers

23,361 views • 1 year ago •via X (Twitter)

Health & Wellness Science & Technology

Anya Rossi• Live Now

Private livecam show

3 Comments

Modular1 year ago

By unlocking optionality across the infrastructure, the application layer, and the hardware, @Modular helped to power the best inference experience for @InworldAI's users through reduced costs and breakthrough latency.

Igor1 year ago

@inworld @OracleCloud Wow!

Phoenix Turing1 year ago

@inworld @OracleCloud @grok summary

Related Videos

Mojo🔥 combines the usability of Python with the performance of C, unlocking unparalleled programmability of AI hardware and extensibility of AI models. Also, it's up to 35000x faster than Python 🤯 and … deploys 🏎

Mojo🔥 combines the usability of Python with the performance of C, unlocking unparalleled programmability of AI hardware and extensibility of AI models. Also, it's up to 35000x faster than Python 🤯 and … deploys 🏎

Modular

124,558 views • 3 years ago

$META is betting on $AMD for AI inference. As AI models become more commoditized, total cost of ownership for token generation matters more than ever. $AMD GPUs are far more cost-efficient than $NVDA for inference. With 3.5B daily users ready to use its AI models, $META is going all-in on $AMD

$META is betting on $AMD for AI inference. As AI models become more commoditized, total cost of ownership for token generation matters more than ever. $AMD GPUs are far more cost-efficient than $NVDA for inference. With 3.5B daily users ready to use its AI models, $META is going all-in on $AMD

Daniel

20,558 views • 8 months ago

“Our stack and models are now actively deployed on AMD.” Aidan Gomez of @Cohere shares how quickly they ported their latest model, CommandA, to the AMD platform and why Instinct GPUs are now their go-to for training at scale. #AdvancingAI

“Our stack and models are now actively deployed on AMD.” Aidan Gomez of @Cohere shares how quickly they ported their latest model, CommandA, to the AMD platform and why Instinct GPUs are now their go-to for training at scale. #AdvancingAI

AMD

21,349 views • 1 year ago

More throughput. Lower token economics. Greater deployment flexibility. Jeff Nalty, Cohere Senior Partnerships Manager, shares why Cohere is seeing strong results with AMD Instinct MI355X Series GPUs, from performance and cost efficiencies to flexible deployment across on-prem, cloud, and hybrid environments.

More throughput. Lower token economics. Greater deployment flexibility. Jeff Nalty, Cohere Senior Partnerships Manager, shares why Cohere is seeing strong results with AMD Instinct MI355X Series GPUs, from performance and cost efficiencies to flexible deployment across on-prem, cloud, and hybrid environments.

AMD

28,912 views • 1 month ago

AWS has announced the general availability of its new Amazon EC2 R8a memory-optimized instances, powered by 5th Gen AMD EPYC processors with clock speeds up to 4.5 GHz. This means more speed and efficiency at a lower total cost of ownership, without having to change their cloud architecture. Read more from Amazon Web Services:

AWS has announced the general availability of its new Amazon EC2 R8a memory-optimized instances, powered by 5th Gen AMD EPYC processors with clock speeds up to 4.5 GHz. This means more speed and efficiency at a lower total cost of ownership, without having to change their cloud architecture. Read more from Amazon Web Services:

AMD EPYC

95,689 views • 11 months ago

Introducing NVIDIA Nemotron 3 Ultra. A frontier smart open model built for long-running agents that need to plan, reason, use tools and keep working across complex coding, research and enterprise workflows. Up to 5x faster inference and up to 30% lower cost for agentic tasks. Learn more:

Introducing NVIDIA Nemotron 3 Ultra. A frontier smart open model built for long-running agents that need to plan, reason, use tools and keep working across complex coding, research and enterprise workflows. Up to 5x faster inference and up to 30% lower cost for agentic tasks. Learn more:

NVIDIA

230,855 views • 4 months ago

🤝 Alphabet and NVIDIA are expanding their decade-long partnership to advance agentic AI, robotics, drug discovery, and more. This involves deep co-engineering with integrated platforms, open-source frameworks, and managed services. ✅ Google Cloud is one of the first to bring the NVIDIA Blackwell platform to the cloud. ✅ Google Distributed Cloud using NVIDIA Confidential Computing on #NVIDIABlackwell for enterprises to run Google Gemini on-premises. ✅ Integration of the NVIDIA AI platform across Vertex AI, Cluster Director and Google Kubernetes Engine. ✅ Bringing the NVIDIA Nemotron family of open models to Vertex AI Model Garden. Watch the full video to learn more ➡️

🤝 Alphabet and NVIDIA are expanding their decade-long partnership to advance agentic AI, robotics, drug discovery, and more. This involves deep co-engineering with integrated platforms, open-source frameworks, and managed services. ✅ Google Cloud is one of the first to bring the NVIDIA Blackwell platform to the cloud. ✅ Google Distributed Cloud using NVIDIA Confidential Computing on #NVIDIABlackwell for enterprises to run Google Gemini on-premises. ✅ Integration of the NVIDIA AI platform across Vertex AI, Cluster Director and Google Kubernetes Engine. ✅ Bringing the NVIDIA Nemotron family of open models to Vertex AI Model Garden. Watch the full video to learn more ➡️

NVIDIA AI Infrastructure

268,567 views • 8 months ago

“Any model, any agent, any accelerator, and any cloud.” Red Hat’s Brian Stevens explains why open AI infrastructure matters as enterprises scale agentic AI workloads, and how Red Hat and AMD are collaborating to help simplify deployment with AMD EPYC processors and AMD Instinct GPUs. Watch the full video on YouTube:

“Any model, any agent, any accelerator, and any cloud.” Red Hat’s Brian Stevens explains why open AI infrastructure matters as enterprises scale agentic AI workloads, and how Red Hat and AMD are collaborating to help simplify deployment with AMD EPYC processors and AMD Instinct GPUs. Watch the full video on YouTube:

AMD

17,208 views • 4 months ago

2 days ago we shipped image generation in <1s 🔥 Today, we make that <300ms 🤯 NVIDIA + AMD⚡️ Full demo below ⬇️

2 days ago we shipped image generation in <1s 🔥 Today, we make that <300ms 🤯 NVIDIA + AMD⚡️ Full demo below ⬇️

Modular

12,173 views • 6 months ago

Generate images in less than 1 second. 99% cheaper than NanoBanna. 🚀 😱 Our latest 26.2 release ships FLUX.2 image generation with a 4.1x speedup over torch.compile on NVIDIA Blackwell - translating to a 5.5x TCO advantage with AMD MI355X. Read more ⬇️

Generate images in less than 1 second. 99% cheaper than NanoBanna. 🚀 😱 Our latest 26.2 release ships FLUX.2 image generation with a 4.1x speedup over torch.compile on NVIDIA Blackwell - translating to a 5.5x TCO advantage with AMD MI355X. Read more ⬇️

Modular

210,658 views • 6 months ago

📢 BREAKING: Nvidia, Microsoft, and Anthropic announce new strategic AI partnerships of roughly $45 Billion. They struck a 3-way deal where Anthropic commits $30B for Azure compute with capacity up to 1GW, and NVIDIA and Microsoft plan to invest up to $10B and $5B in Anthropic. Claude will scale on Azure using NVIDIA systems, which expands access for enterprises and makes Claude available across the major clouds. Anthropic and NVIDIA will co-design to tune both model stacks and future chips for performance, efficiency, and total cost of ownership, so the models run faster per dollar. The first wave targets NVIDIA Grace Blackwell and Vera Rubin systems, which pair high-bandwidth memory with dense interconnect to push training and inference throughput. Microsoft will expose Anthropic models in Azure AI Foundry, including Claude Sonnet 4.5, Claude Opus 4.1, and Claude Haiku 4.5, so teams can pick latency and cost profiles. Claude will also remain available across the Copilot family, which means coding, document automation, and workflow agents can call the same model line. The 1GW commitment signals many thousands of GPUs worth of capacity, which reduces queue times and stabilizes unit economics for long-running training runs. The joint optimization loop, models shaping chips and chips shaping models, typically yields double-digit efficiency gains over off-the-shelf deployments.

📢 BREAKING: Nvidia, Microsoft, and Anthropic announce new strategic AI partnerships of roughly $45 Billion. They struck a 3-way deal where Anthropic commits $30B for Azure compute with capacity up to 1GW, and NVIDIA and Microsoft plan to invest up to $10B and $5B in Anthropic. Claude will scale on Azure using NVIDIA systems, which expands access for enterprises and makes Claude available across the major clouds. Anthropic and NVIDIA will co-design to tune both model stacks and future chips for performance, efficiency, and total cost of ownership, so the models run faster per dollar. The first wave targets NVIDIA Grace Blackwell and Vera Rubin systems, which pair high-bandwidth memory with dense interconnect to push training and inference throughput. Microsoft will expose Anthropic models in Azure AI Foundry, including Claude Sonnet 4.5, Claude Opus 4.1, and Claude Haiku 4.5, so teams can pick latency and cost profiles. Claude will also remain available across the Copilot family, which means coding, document automation, and workflow agents can call the same model line. The 1GW commitment signals many thousands of GPUs worth of capacity, which reduces queue times and stabilizes unit economics for long-running training runs. The joint optimization loop, models shaping chips and chips shaping models, typically yields double-digit efficiency gains over off-the-shelf deployments.

Rohan Paul

39,781 views • 10 months ago

Modular has acquired BentoML! 🤝 10K+ orgs use BentoML for production AI, including 50+ Fortune 500 companies. We're pairing their deployment platform with MAX + Mojo's hardware optimization. BentoML stays open source (Apache 2.0), and we’re doubling down on OSS in 2026. Ask BentoML founder Chaoyu Yang and Chris Lattner anything on Feb 17 at 9:30am PT. Get all the details:

Modular has acquired BentoML! 🤝 10K+ orgs use BentoML for production AI, including 50+ Fortune 500 companies. We're pairing their deployment platform with MAX + Mojo's hardware optimization. BentoML stays open source (Apache 2.0), and we’re doubling down on OSS in 2026. Ask BentoML founder Chaoyu Yang and Chris Lattner anything on Feb 17 at 9:30am PT. Get all the details:

Modular

28,702 views • 7 months ago

📈 NVIDIA tops AI leaderboards and benchmarks with open models driven by extreme co-design across compute, networking, memory, storage, and software. This includes models for biology, AI physics, agentic AI, physical AI, robotics, and autonomous vehicles. By being vertically integrated and horizontally open, NVIDIA enables its ecosystem to scale agentic workflows at every layer of the stack with the lowest cost per token.

📈 NVIDIA tops AI leaderboards and benchmarks with open models driven by extreme co-design across compute, networking, memory, storage, and software. This includes models for biology, AI physics, agentic AI, physical AI, robotics, and autonomous vehicles. By being vertically integrated and horizontally open, NVIDIA enables its ecosystem to scale agentic workflows at every layer of the stack with the lowest cost per token.

NVIDIA

85,822 views • 5 months ago

Some time ago, I had the idea to port NVIDIA Physical AI stack to AMD. The motivation was to improve hardware diversity and enable world models and VLAs to run beyond a single ecosystem. We started with NVIDIA Cosmos Predict 2.5-2B. Porting wasn’t trivial: these models are deeply optimized for NVIDIA’s stack. We used this as an opportunity to apply our ROCm kernels. The results were surprising: Both encode and diffusion run faster on AMD Instinct MI300X vs. NVIDIA H200 (FA3) and we still saw significant headroom for further optimization. Quality is unchanged across modalities (validated with WorldJen) To be clear, this is no luck. We have deep experience with diffusion models and AMD GPUs. But this just gives us a good opportunity to get closer to a true hardware-to-hardware comparison, as we work with less software abstractions than usual. Just to give an example, on AMD, memory instructions are async with a hardware queue of ordered pending instructions, enabling concurrent load/store with compute without warp specialization. Bottom line: there are real architectural advantages on AMD, if you take the time to work with the hardware. Note, we did tradeoff ~20% higher memory usage, That being said, AMD has more to give to begin with :) in the coming weeks: AMD versions of Cosmos Transfer and GR00T, an even faster version of Cosmos Predict, and open-sourcing an attention kernel faster than AITER v3 (which is closed-source for some reason? cc: Anush Elangovan )

Some time ago, I had the idea to port NVIDIA Physical AI stack to AMD. The motivation was to improve hardware diversity and enable world models and VLAs to run beyond a single ecosystem. We started with NVIDIA Cosmos Predict 2.5-2B. Porting wasn’t trivial: these models are deeply optimized for NVIDIA’s stack. We used this as an opportunity to apply our ROCm kernels. The results were surprising: Both encode and diffusion run faster on AMD Instinct MI300X vs. NVIDIA H200 (FA3) and we still saw significant headroom for further optimization. Quality is unchanged across modalities (validated with WorldJen) To be clear, this is no luck. We have deep experience with diffusion models and AMD GPUs. But this just gives us a good opportunity to get closer to a true hardware-to-hardware comparison, as we work with less software abstractions than usual. Just to give an example, on AMD, memory instructions are async with a hardware queue of ordered pending instructions, enabling concurrent load/store with compute without warp specialization. Bottom line: there are real architectural advantages on AMD, if you take the time to work with the hardware. Note, we did tradeoff ~20% higher memory usage, That being said, AMD has more to give to begin with :) in the coming weeks: AMD versions of Cosmos Transfer and GR00T, an even faster version of Cosmos Predict, and open-sourcing an attention kernel faster than AITER v3 (which is closed-source for some reason? cc: Anush Elangovan )

Omer Shlomovits

36,648 views • 5 months ago

The #NVIDIAVeraRubin platform consists of five rack-scale systems for AI agents with a supply chain spanning over 350 factory sites across 30 countries with millions square feet of factory floor space. Vera Rubin is in full production, co-designed with NVIDIA DSX and DSX MaxLPS to deliver the lowest token cost and maximum tokens per watt. Congratulations to our partners who have their engineering racks up and running: CoreWeave, Dell Technologies, Microsoft, and Oracle Cloud.

The #NVIDIAVeraRubin platform consists of five rack-scale systems for AI agents with a supply chain spanning over 350 factory sites across 30 countries with millions square feet of factory floor space. Vera Rubin is in full production, co-designed with NVIDIA DSX and DSX MaxLPS to deliver the lowest token cost and maximum tokens per watt. Congratulations to our partners who have their engineering racks up and running: CoreWeave, Dell Technologies, Microsoft, and Oracle Cloud.

NVIDIA AI Infrastructure

16,931 views • 2 months ago

Recraft is building the next generation of AI image models for creative teams. With Nebius AI Cloud, the team gets scalable infrastructure, early access to the latest NVIDIA GPUs and the reliability to serve millions of users worldwide.

Recraft is building the next generation of AI image models for creative teams. With Nebius AI Cloud, the team gets scalable infrastructure, early access to the latest NVIDIA GPUs and the reliability to serve millions of users worldwide.

Nebius

12,288,315 views • 2 months ago

“The Coatue team mapped out cloud revenue market share, Oracle at 5%, Amazon at 44% with AWS, Google 19%, Microsoft 30%. Next to that, they showed NVIDIA GPU allocation: Microsoft and Google match their cloud share. Amazon is 44% cloud revenue but only 20% GPUs. Oracle jumps from 5% to 19%. Cori comes out of nowhere at 11%.” “One obvious takeaway is that Amazon has half the share of GPUs than their share of AWS. So that could mean one of two things, either AWS is behind in AI, that could be one, or they're pursuing a different hardware strategy than its competitors.” — Bill Gurley Bill Gurley and Thomas Laffont Thomas Laffont

“The Coatue team mapped out cloud revenue market share, Oracle at 5%, Amazon at 44% with AWS, Google 19%, Microsoft 30%. Next to that, they showed NVIDIA GPU allocation: Microsoft and Google match their cloud share. Amazon is 44% cloud revenue but only 20% GPUs. Oracle jumps from 5% to 19%. Cori comes out of nowhere at 11%.” “One obvious takeaway is that Amazon has half the share of GPUs than their share of AWS. So that could mean one of two things, either AWS is behind in AI, that could be one, or they're pursuing a different hardware strategy than its competitors.” — Bill Gurley Bill Gurley and Thomas Laffont Thomas Laffont

bg2clips

180,310 views • 1 year ago

Our cofounder tim built an AI storybook app using Black Forest Labs' FLUX2 and Google Gemma 4 on Modular Cloud. Pick a character, make choices, and the story branches endlessly, with every page written and illustrated in real time. Tim has spent his career obsessing over inference latency, first at Google, now at Modular. Building something his kids use settled it: in a real-time generative app, the inference platform determines the experience as much as the model. The numbers back that up. From 24 hours of production traffic: first prose in 420ms, a full illustration in under 6 seconds, 85% of page turns in 48ms. Create your own story with Inkwell and share it. We're sending swag to our favorites:

Our cofounder tim built an AI storybook app using Black Forest Labs' FLUX2 and Google Gemma 4 on Modular Cloud. Pick a character, make choices, and the story branches endlessly, with every page written and illustrated in real time. Tim has spent his career obsessing over inference latency, first at Google, now at Modular. Building something his kids use settled it: in a real-time generative app, the inference platform determines the experience as much as the model. The numbers back that up. From 24 hours of production traffic: first prose in 420ms, a full illustration in under 6 seconds, 85% of page turns in 48ms. Create your own story with Inkwell and share it. We're sending swag to our favorites:

Modular

14,324 views • 4 months ago

Introducing Project Ceiba, a groundbreaking collaboration between #AWS and NVIDIA, that aims to push the boundaries of #AI by constructing the largest AI supercomputer in the cloud. ☁️ ⚙️ 💻 Learn more. ➡️

Introducing Project Ceiba, a groundbreaking collaboration between #AWS and NVIDIA, that aims to push the boundaries of #AI by constructing the largest AI supercomputer in the cloud. ☁️ ⚙️ 💻 Learn more. ➡️

Amazon Web Services

16,205 views • 2 years ago