Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Transformer architecture can be difficult to conceptualize. This clip makes it considerably more accessible. At the International Semiconductor Industry Group (ISIG) Executive Summit, Marvell President of the Data Center Group Sandeep Bharathi walks through how large language models process information using a restaurant analogy, from the moment an order... show more

Marvell Technology

15,228 subscribers

11,026 views • 1 month ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Chris Camillo explains how Bloom Energy is solving AI data center energy usage problems… “ $BE is one of the only companies in the world that has a method to essentially bring almost an infinite amount of energy to these data centers." And we won't get into all the technology, but they are now working with hyperscalers and they are a resource for energy that will allow a data center to get ramped up quicker. "So, even though it costs a little bit more money today to put an energy cell from Bloom, an energy box from Bloom, which are energy cells, it essentially takes things like natural gas, but instead of burning the natural gas, there is a chemical process that turns that natural gas into energy for the data center."

Chris Camillo explains how Bloom Energy is solving AI data center energy usage problems… “ $BE is one of the only companies in the world that has a method to essentially bring almost an infinite amount of energy to these data centers." And we won't get into all the technology, but they are now working with hyperscalers and they are a resource for energy that will allow a data center to get ramped up quicker. "So, even though it costs a little bit more money today to put an energy cell from Bloom, an energy box from Bloom, which are energy cells, it essentially takes things like natural gas, but instead of burning the natural gas, there is a chemical process that turns that natural gas into energy for the data center."

The Iced Coffee Hour

274,893 views • 2 months ago

A massive 350,000 square foot data center in Massachusetts is nicknamed “The Dungeon” by residents The data center promised it would provide lots of new jobs to the community. Only about 12 people work there…. The data center is causing constant visible pollution and a lot of noise to residents It’s using about 120,000 gallons of water per day and using as much power as 40,000 American homes Residents were lied to Get this, the data center is buying nearby homes and properties to expand….

A massive 350,000 square foot data center in Massachusetts is nicknamed “The Dungeon” by residents The data center promised it would provide lots of new jobs to the community. Only about 12 people work there…. The data center is causing constant visible pollution and a lot of noise to residents It’s using about 120,000 gallons of water per day and using as much power as 40,000 American homes Residents were lied to Get this, the data center is buying nearby homes and properties to expand….

Wall Street Apes

105,431 views • 13 days ago

In the next 2-3 years, the skills you need to build an ETL process will fundamentally differ from anything we've done in the last decade. The EXTRACT and LOAD steps of an ETL pipeline have been solved forever. They are boring, and I suspect they will continue to be boring. But the TRANSFORM step is where the money is at (and where innovation will continue happening!) An ETL pipeline that can use Large Language Models effectively to transform data is a whole new ballgame. I recorded a video to talk about this. And more importantly, I talk about how you can build ETL pipelines to transform UNSTRUCTURED data (the kind of data that's really hard to handle).

In the next 2-3 years, the skills you need to build an ETL process will fundamentally differ from anything we've done in the last decade. The EXTRACT and LOAD steps of an ETL pipeline have been solved forever. They are boring, and I suspect they will continue to be boring. But the TRANSFORM step is where the money is at (and where innovation will continue happening!) An ETL pipeline that can use Large Language Models effectively to transform data is a whole new ballgame. I recorded a video to talk about this. And more importantly, I talk about how you can build ETL pipelines to transform UNSTRUCTURED data (the kind of data that's really hard to handle).

Santiago

56,412 views • 1 year ago

$MU $00060 $00530 $TSM Evolution of the AI Memory Hierarchy: HBM, DRAM, & SRAM Overview and Market Analysis The video outlines the strategic shift of modern computing toward a memory-centric architecture driven by the demands of generative AI. It explains that the industry is moving away from a traditional commodity model to a tightly coupled hierarchy of specialized technologies, where High Bandwidth Memory (HBM) and advanced packaging act as primary bottlenecks for performance. Leading manufacturers like SK hynix, Samsung, and Micron are navigating complex transitions to HBM4 and CXL-based expansion to overcome data movement constraints. Beyond the memory producers, the ecosystem relies on integration enablers like TSMC for packaging and specialized equipment providers for high-precision manufacturing. Ultimately, the analysis suggests that memory intensity and power efficiency have become the decisive factors in the success of gigawatt-scale data centers.

$MU $00060 $00530 $TSM Evolution of the AI Memory Hierarchy: HBM, DRAM, & SRAM Overview and Market Analysis The video outlines the strategic shift of modern computing toward a memory-centric architecture driven by the demands of generative AI. It explains that the industry is moving away from a traditional commodity model to a tightly coupled hierarchy of specialized technologies, where High Bandwidth Memory (HBM) and advanced packaging act as primary bottlenecks for performance. Leading manufacturers like SK hynix, Samsung, and Micron are navigating complex transitions to HBM4 and CXL-based expansion to overcome data movement constraints. Beyond the memory producers, the ecosystem relies on integration enablers like TSMC for packaging and specialized equipment providers for high-precision manufacturing. Ultimately, the analysis suggests that memory intensity and power efficiency have become the decisive factors in the success of gigawatt-scale data centers.

TheValueist

23,901 views • 6 months ago

🚀 NASA and Microsoft have joined forces to create the Earth Copilot, an AI tool which takes the more than 100 petabytes of data that NASA’s satellites capture and makes it accessible to anyone using natural language for the first time. #MSIgnite

🚀 NASA and Microsoft have joined forces to create the Earth Copilot, an AI tool which takes the more than 100 petabytes of data that NASA’s satellites capture and makes it accessible to anyone using natural language for the first time. #MSIgnite

Microsoft

72,753 views • 1 year ago

Lutnick: If you are building a data center, you were going to give power to the grid and drive the energy costs of everybody around that data center lower. Trump is going to bring lower energy prices. Everywhere that a big company builds a big data center, they are going to lower the price of energy in the entire area around it.

Lutnick: If you are building a data center, you were going to give power to the grid and drive the energy costs of everybody around that data center lower. Trump is going to bring lower energy prices. Everywhere that a big company builds a big data center, they are going to lower the price of energy in the entire area around it.

Acyn

494,169 views • 6 months ago

MiniMax is building with next-gen AGI, offering foundation AI models and revolutionizing content creation with AI-powered video, music, chat products. With Alibaba's one-stop multimodal data platform, MiniMax can save the complexity of handling multimodal data work with the AI-ready data foundation, and boost streamlined innovation on it. Feifei Li, SVP and President of International Business at Alibaba Cloud Intelligence Group, shares how the right tech stack can elevate AI from potential to powerful. 🚀

MiniMax is building with next-gen AGI, offering foundation AI models and revolutionizing content creation with AI-powered video, music, chat products. With Alibaba's one-stop multimodal data platform, MiniMax can save the complexity of handling multimodal data work with the AI-ready data foundation, and boost streamlined innovation on it. Feifei Li, SVP and President of International Business at Alibaba Cloud Intelligence Group, shares how the right tech stack can elevate AI from potential to powerful. 🚀

Alibaba Group

305,831 views • 7 months ago

A Heavy Lift for a Powerful Future! $IREN's multi-disciplinary engineering teams successfully installed the Childress Phase 4 substation transformer over the weekend, bringing the company one step closer to energizing an additional 150MW of data center capacity. David Shaw, Chief Operating Officer, commented: "With unprecedented growth in the data center industry and increasingly constrained supply chains, careful and proactive planning is more critical than ever. The ahead-of-schedule installation of this 138/34.5 kV transformer underscores the strategic importance of IREN’s internal procurement team and strength of our supplier partnerships."

A Heavy Lift for a Powerful Future! $IREN's multi-disciplinary engineering teams successfully installed the Childress Phase 4 substation transformer over the weekend, bringing the company one step closer to energizing an additional 150MW of data center capacity. David Shaw, Chief Operating Officer, commented: "With unprecedented growth in the data center industry and increasingly constrained supply chains, careful and proactive planning is more critical than ever. The ahead-of-schedule installation of this 138/34.5 kV transformer underscores the strategic importance of IREN’s internal procurement team and strength of our supplier partnerships."

IREN

60,064 views • 1 year ago

$NVDA CEO Jensen Huang saying $MRVL can become “the next trillion-dollar company” doesn’t sound crazy when you realize it sits directly inside one of the biggest bottlenecks of the AI buildout: networking. Marvell is trying to become the connective tissue of the AI factory with exposure to custom ASICs, optics, switching, photonics and data movement layer every large-scale AI system needs.

$NVDA CEO Jensen Huang saying $MRVL can become “the next trillion-dollar company” doesn’t sound crazy when you realize it sits directly inside one of the biggest bottlenecks of the AI buildout: networking. Marvell is trying to become the connective tissue of the AI factory with exposure to custom ASICs, optics, switching, photonics and data movement layer every large-scale AI system needs.

Shay Boloor

533,096 views • 1 month ago

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 views • 9 days ago

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:

Andrew Ng

253,812 views • 1 year ago

Threadguy explains why Apple will capture ALL of the value from AI “Models have become a commodity, we'll be able to just use the best model at any given time” "The more data AI has access to, the better it is. Everyone already has an iPhone, the next iPhone is just going to launch with some OpenClaw esque assistant utilising the best model plugged into it and its probably going to be the best AI experience you’ve ever had”

Threadguy explains why Apple will capture ALL of the value from AI “Models have become a commodity, we'll be able to just use the best model at any given time” "The more data AI has access to, the better it is. Everyone already has an iPhone, the next iPhone is just going to launch with some OpenClaw esque assistant utilising the best model plugged into it and its probably going to be the best AI experience you’ve ever had”

Jack

24,179 views • 4 months ago

This is the best way to understand how ML models actually work! Use Drawdata to draw a 2D dataset in Jupyter. Use it to actively pick data from the widget and update the model as the data is being drawn! Fully interactive, real-time, and open-source!

This is the best way to understand how ML models actually work! Use Drawdata to draw a 2D dataset in Jupyter. Use it to actively pick data from the widget and update the model as the data is being drawn! Fully interactive, real-time, and open-source!

Daily Dose of Data Science

52,070 views • 8 months ago

IEEE recently recognized the 40th anniversary of the FPGA at an event at AMD in San Jose. When it made its debut four decades ago, the FPGA introduced a new kind of flexibility in hardware that has reshaped the semiconductor industry. The IEEE event honored AMD and the engineers, partners, and customers who made it possible. It also paid tribute to the decades of innovation and collaboration that continue to power innovation across so many applications, from communications, automotive, and industrial systems, to aerospace, data centers, and physical AI systems at the edge.

IEEE recently recognized the 40th anniversary of the FPGA at an event at AMD in San Jose. When it made its debut four decades ago, the FPGA introduced a new kind of flexibility in hardware that has reshaped the semiconductor industry. The IEEE event honored AMD and the engineers, partners, and customers who made it possible. It also paid tribute to the decades of innovation and collaboration that continue to power innovation across so many applications, from communications, automotive, and industrial systems, to aerospace, data centers, and physical AI systems at the edge.

AMD Embedded

11,158 views • 3 months ago

When an anonymous tech giant got St. Charles to sign NDAs to ram through a data center, the Missouri city rose up to beat the project back. Now the city, not half an hour from St. Louis, has enacted the first indefinite data center ban in the country. Here's how they did it:

When an anonymous tech giant got St. Charles to sign NDAs to ram through a data center, the Missouri city rose up to beat the project back. Now the city, not half an hour from St. Louis, has enacted the first indefinite data center ban in the country. Here's how they did it:

More Perfect Union

29,957 views • 1 month ago

Enrollment is now open for the Data Engineering Professional Certificate! Data engineers are the architects of modern organizations, ensuring data is reliable, accessible, and ready for analytics and machine learning. This professional certificate is tailored to equip you with the critical skills, through frameworks and hands-on practice, to excel in this role. Taught by industry expert Joe Reis, co-author of the best-selling book "Fundamentals of Data Engineering," along with 17 guest instructors from the data field, you will gain expertise to start and further your career in the high-demand field of data engineering. Key focus areas: 🗂️ Data Engineering Lifecycle: Learn the important stages of building an efficient data pipeline that creates business value. 📥 Data Ingestion: Learn how to efficiently gather data from various sources. 💾 Data Storage: Master the techniques for storing data securely and cost-effectively. 🔄 Data Transformation: Understand how to clean, organize, and prepare data for analysis and machine learning. 🏗️ Data Architecture Design: Build robust architectures that support scalable, efficient data workflows. 📊 Serving Data: Ensure that data is available to stakeholders when and where they need it to drive business decisions. Enroll now!

Enrollment is now open for the Data Engineering Professional Certificate! Data engineers are the architects of modern organizations, ensuring data is reliable, accessible, and ready for analytics and machine learning. This professional certificate is tailored to equip you with the critical skills, through frameworks and hands-on practice, to excel in this role. Taught by industry expert Joe Reis, co-author of the best-selling book "Fundamentals of Data Engineering," along with 17 guest instructors from the data field, you will gain expertise to start and further your career in the high-demand field of data engineering. Key focus areas: 🗂️ Data Engineering Lifecycle: Learn the important stages of building an efficient data pipeline that creates business value. 📥 Data Ingestion: Learn how to efficiently gather data from various sources. 💾 Data Storage: Master the techniques for storing data securely and cost-effectively. 🔄 Data Transformation: Understand how to clean, organize, and prepare data for analysis and machine learning. 🏗️ Data Architecture Design: Build robust architectures that support scalable, efficient data workflows. 📊 Serving Data: Ensure that data is available to stakeholders when and where they need it to drive business decisions. Enroll now!

DeepLearning.AI

20,833 views • 1 year ago

The 2027 NDAA goes much further than merging America’s military with Israel’s. It prioritizes giving Israel access to American AI, American data, and American autonomous weapons systems under the guise of mutual defense. Your local data center is an Israeli surveillance center.

The 2027 NDAA goes much further than merging America’s military with Israel’s. It prioritizes giving Israel access to American AI, American data, and American autonomous weapons systems under the guise of mutual defense. Your local data center is an Israeli surveillance center.

Frankie Stockes

144,115 views • 1 month ago

The lack of understanding of how current AI/ large language models work makes us susceptible to thinking that AI is more powerful and all-knowing than it actually is. That’s what concerns me the most.

The lack of understanding of how current AI/ large language models work makes us susceptible to thinking that AI is more powerful and all-knowing than it actually is. That’s what concerns me the most.

Sinead Bovell

143,642 views • 3 years ago