Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Training LFMs at scale means solving parallelism across every layer of the architecture. And not all layers are the same. Our CTO Mathias Lechner (Mathias Lechner) sits down with Liquid's founding engineer Paul Pak (Paul Pak) to talk training infrastructure: Data, tensor, pipeline, expert, and context parallelism, and how... show more

Liquid AI

31,742 subscribers

17,908 просмотров • 2 месяцев назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Building a model is just the start. Post-training makes it useful. Our CTO Mathias Lechner (Mathias Lechner) sits down for a conversation with Maxime Labonne (Maxime Labonne), our head of post-training, on the pipeline that takes a base model from autocomplete to something that can reason and follow instructions.

Liquid AI

26,121 просмотров • 3 месяцев назад

Most multimodal systems need data that combines every modality together. Hard to get, expensive to build. Our CTO Mathias Lechner, Mathias Lechner, sits down with Saniya Karwa, Saniya Karwa, from our multimodal research team to talk about building a mode that handles text, audio, and image, and why you might not need as much combined training data as you think.

Most multimodal systems need data that combines every modality together. Hard to get, expensive to build. Our CTO Mathias Lechner, Mathias Lechner, sits down with Saniya Karwa, Saniya Karwa, from our multimodal research team to talk about building a mode that handles text, audio, and image, and why you might not need as much combined training data as you think.

Liquid AI

15,382 просмотров • 2 месяцев назад

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!

Andrew Ng

101,794 просмотров • 4 месяцев назад

🚀 Excited to release LongLive 2.0! 🎬 An end-to-end infrastructure for long video generation, with FP4 and parallelism at the core of both training and inference. ⚡45.7 FPS generation speed on 5B model⚡ ✨ LongLive 2.0 supports real-video training, few-step distillation, multi-shot training/inference, sequence-parallel acceleration, NVFP4 KV cache, and async VAE decoding deployment. 🧩 To our knowledge, this is the first open-source 4-bit long video generation infra that covers both training and inference. 🙌 Welcome to check it out, try it, and share feedback! 🔗 Code: 📰 Paper: 🎥 Demo: #LongVideoGeneration #VideoGeneration #Realtime #AIInfra #EfficientAI #FP4 #Parallel #NVIDIA

🚀 Excited to release LongLive 2.0! 🎬 An end-to-end infrastructure for long video generation, with FP4 and parallelism at the core of both training and inference. ⚡45.7 FPS generation speed on 5B model⚡ ✨ LongLive 2.0 supports real-video training, few-step distillation, multi-shot training/inference, sequence-parallel acceleration, NVFP4 KV cache, and async VAE decoding deployment. 🧩 To our knowledge, this is the first open-source 4-bit long video generation infra that covers both training and inference. 🙌 Welcome to check it out, try it, and share feedback! 🔗 Code: 📰 Paper: 🎥 Demo: #LongVideoGeneration #VideoGeneration #Realtime #AIInfra #EfficientAI #FP4 #Parallel #NVIDIA

Yukang Chen

59,546 просмотров • 3 месяцев назад

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically *retrieved* “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are *not* collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant *sub-trajectories* from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically retrieved “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are not collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant sub-trajectories from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

Abhishek Gupta

12,045 просмотров • 1 год назад

𝐓𝐡𝐞 𝐒𝐀𝐏 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐂𝐞𝐧𝐭𝐞𝐫 𝐢𝐬 𝐧𝐨𝐰 𝐥𝐢𝐯𝐞 𝐨𝐧 𝐭𝐡𝐞 𝐒𝐀𝐏 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐂𝐞𝐧𝐭𝐞𝐫: More details in the post Built by our team at the Office of the CTO, SAP, the Architecture Center is designed to scale how we build, publish, and govern reference solution architectures across the SAP ecosystem, with open collaboration at its core. Anirban Navya Khurana SAP LABS INDIA Sindhu Gangadharan

𝐓𝐡𝐞 𝐒𝐀𝐏 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐂𝐞𝐧𝐭𝐞𝐫 𝐢𝐬 𝐧𝐨𝐰 𝐥𝐢𝐯𝐞 𝐨𝐧 𝐭𝐡𝐞 𝐒𝐀𝐏 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐂𝐞𝐧𝐭𝐞𝐫: More details in the post Built by our team at the Office of the CTO, SAP, the Architecture Center is designed to scale how we build, publish, and govern reference solution architectures across the SAP ecosystem, with open collaboration at its core. Anirban Navya Khurana SAP LABS INDIA Sindhu Gangadharan

PVN PavanKumar

251,687 просмотров • 1 год назад

Agentic advertising isn't possible without a single source of truth. Every agent, every trade, every settlement has to resolve against the same record, or the whole thing collapses back into the mess we're leaving behind. Sui and Walrus 🦭/acc are how we get there. The settlement layer and the data layer that make agent-to-agent advertising actually workable at scale. Labs is the community heart of all of this. Built with you, and the best part is still ahead.

Agentic advertising isn't possible without a single source of truth. Every agent, every trade, every settlement has to resolve against the same record, or the whole thing collapses back into the mess we're leaving behind. Sui and Walrus 🦭/acc are how we get there. The settlement layer and the data layer that make agent-to-agent advertising actually workable at scale. Labs is the community heart of all of this. Built with you, and the best part is still ahead.

Alkimi

10,890 просмотров • 2 месяцев назад

BitGo sits down with Ian Kane, Head of Partnerships at Midnight, to learn how they are building the next generation of blockchain infrastructure. Privacy, compliance, and institutional scale come together in a new approach to moving assets onchain.

BitGo sits down with Ian Kane, Head of Partnerships at Midnight, to learn how they are building the next generation of blockchain infrastructure. Privacy, compliance, and institutional scale come together in a new approach to moving assets onchain.

BitGo

58,857 просмотров • 1 год назад

Our co-founder Pratyush Kumar highlighted Sarvam's "contrarian bet" of being a full-stack AI platform. That means we build every layer: the models, how they run (inference), how agents work with each other (orchestration), and the AI applications people use. This lets us build AI that actually works for India's unique context and diverse language needs.

Our co-founder Pratyush Kumar highlighted Sarvam's "contrarian bet" of being a full-stack AI platform. That means we build every layer: the models, how they run (inference), how agents work with each other (orchestration), and the AI applications people use. This lets us build AI that actually works for India's unique context and diverse language needs.

Sarvam AI

16,310 просмотров • 8 месяцев назад

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables

Noisy

48,697 просмотров • 2 месяцев назад

From training to inference at massive scale, delivering on the promise of AI depends on collaboration across open hardware and software. AMD and OpenAI are working together to tailor hardware like Instinct MI455X and Helios for where AI workloads are going, not just where they are today. Greg Brockman (Greg Brockman) on the power of our engineering partnership at #CES2026.

From training to inference at massive scale, delivering on the promise of AI depends on collaboration across open hardware and software. AMD and OpenAI are working together to tailor hardware like Instinct MI455X and Helios for where AI workloads are going, not just where they are today. Greg Brockman (Greg Brockman) on the power of our engineering partnership at #CES2026.

AMD

20,021 просмотров • 7 месяцев назад

Don't waste 2 years learning how Claude and ChatGPT actually work. Stanford just dropped a 1-hour course on the exact pipeline behind them. 0:00 - policy gradient basics 23:02 - PPO for LLM training 55:28 - how models learn chain of thought 1:02:50 - the architecture, explained look at the third one, because that is the part people mistake for a prompting trick. Task → Steps → Check → Answer chain of thought is not something you talk a model into. it is trained in, by rewarding runs that broke the work into steps. that decomposition happens inside one context window. doing it across many is a different problem, and training does not solve it for you. training gives you a better node. what you do with a hundred of them is a different job. Anthropic pays $750,000 a year to engineers who work at this layer. watch it today, then read the graph engineering course below.

Don't waste 2 years learning how Claude and ChatGPT actually work. Stanford just dropped a 1-hour course on the exact pipeline behind them. 0:00 - policy gradient basics 23:02 - PPO for LLM training 55:28 - how models learn chain of thought 1:02:50 - the architecture, explained look at the third one, because that is the part people mistake for a prompting trick. Task → Steps → Check → Answer chain of thought is not something you talk a model into. it is trained in, by rewarding runs that broke the work into steps. that decomposition happens inside one context window. doing it across many is a different problem, and training does not solve it for you. training gives you a better node. what you do with a hundred of them is a different job. Anthropic pays $750,000 a year to engineers who work at this layer. watch it today, then read the graph engineering course below.

Hanako

21,564 просмотров • 17 дней назад

Visa and WeFi Talk: The Future of Stablecoin Payment. Our first episode is coming next week, and we sat down with Visa to talk about where global payments go next. The conversation features Alexandra Soroko, Growth Product & Partnerships at Visa, and Michael Batuev, Head of Global Payments at WeFi. We discuss how global payment infrastructure evaluates stablecoin integration, what real operational deployment involves, and how financial systems at scale decide when new rails become part of core infrastructure. WeFi is not watching the future unfold. We are in the room helping shape it. This is the first of many conversations we will host with major players across the industry.

Visa and WeFi Talk: The Future of Stablecoin Payment. Our first episode is coming next week, and we sat down with Visa to talk about where global payments go next. The conversation features Alexandra Soroko, Growth Product & Partnerships at Visa, and Michael Batuev, Head of Global Payments at WeFi. We discuss how global payment infrastructure evaluates stablecoin integration, what real operational deployment involves, and how financial systems at scale decide when new rails become part of core infrastructure. WeFi is not watching the future unfold. We are in the room helping shape it. This is the first of many conversations we will host with major players across the industry.

WeFi

24,926 просмотров • 6 месяцев назад

Great to meet with Beca in Auckland and see how they are using Azure, Foundry, and their BEYON platform to make the New Zealand Geotechnical Database more accessible and useful. It is a powerful example of AI helping engineers access critical data faster and make better decisions, as they build more resilient infrastructure across New Zealand.

Great to meet with Beca in Auckland and see how they are using Azure, Foundry, and their BEYON platform to make the New Zealand Geotechnical Database more accessible and useful. It is a powerful example of AI helping engineers access critical data faster and make better decisions, as they build more resilient infrastructure across New Zealand.

Satya Nadella

101,177 просмотров • 4 месяцев назад

We think #GenAI will transform virtually every customer experience. That’s why we’re doubling down on all three layers of what’s called the GenAI “stack.” What do we mean by stack? Simple. Three layers, bottom to top. The “bottom layer” means the infrastructure. Think chips, servers, networking – the software, hardware, and computing power in our data centers used to train, build, and deploy AI models and applications. The “middle layer” is the GenAI sandbox – all the tools, Large Language Models (LLMs), and Foundation Models (FMs) we’ve built into Amazon Bedrock. This lets customers pick the #AI models that best fit their needs to easily and securely customize GenAI applications specific to their business. The “top layer” means the popular agents and assistants that people actually interact with – now including Amazon Q. These are built from the models in the middle layer – in Amazon Bedrock. While other apps might help you write a quick email, or edit an image, Amazon Q’s meant for taking care of business. It’s been engineered to be the most capable GenAI-powered assistant for speeding up software dev and leveraging companies’ internal data. Are you as bullish on the future of GenAI as we are? Listen to our CEO Andy Jassy talk about our investment across the stack.

We think #GenAI will transform virtually every customer experience. That’s why we’re doubling down on all three layers of what’s called the GenAI “stack.” What do we mean by stack? Simple. Three layers, bottom to top. The “bottom layer” means the infrastructure. Think chips, servers, networking – the software, hardware, and computing power in our data centers used to train, build, and deploy AI models and applications. The “middle layer” is the GenAI sandbox – all the tools, Large Language Models (LLMs), and Foundation Models (FMs) we’ve built into Amazon Bedrock. This lets customers pick the #AI models that best fit their needs to easily and securely customize GenAI applications specific to their business. The “top layer” means the popular agents and assistants that people actually interact with – now including Amazon Q. These are built from the models in the middle layer – in Amazon Bedrock. While other apps might help you write a quick email, or edit an image, Amazon Q’s meant for taking care of business. It’s been engineered to be the most capable GenAI-powered assistant for speeding up software dev and leveraging companies’ internal data. Are you as bullish on the future of GenAI as we are? Listen to our CEO Andy Jassy talk about our investment across the stack.

Amazon

47,421 просмотров • 2 лет назад

New short course: Build Long-Context AI Apps with Jamba. Learn about state space models (SSMs), which have emerged as an alternative to transformers! Specifically, Jamba is a hybrid transformer-Mamba architecture that combines strengths of the transformer with ideas from SSMs. This course is built with AI21 Labs and taught by Chen Wang and Chen Almagor. The transformer architecture is computationally expensive when handling very long input contexts. But there's an alternative called Mamba, a selective state space model that can process very long contexts with a much lower computational cost. However, researchers found that the pure Mamba architecture underperforms in understanding the context, and gives lower-quality responses. To overcome this, AI21 developed the Jamba model, which combines Mamba's computational efficiency with the transformer's attention mechanism to help with the output quality. In this course, you’ll learn about how state space models, and Jamba, work. You’ll also learn how to prompt Jamba, use it to process long documents, and build long-context RAG apps. - Learn how Jamba combines transformer and state space model architectures to achieve high performance and quality - Use the AI21 SDK, with an example of prompting over a large 200k-token annual financial report of Nvidia - Use Jamba for tool-calling, with hands-on examples from calling simple arithmetic calculations to a function that returns quarterly company financial reports. - Learn how training for long context is done, and the metrics used for its evaluation - Create a RAG app using the AI21 Conversational RAG tool and build your own RAG pipeline that uses Jamba and LangChain. By the end of this course, you'll learn how to build applications that can handle context as long as an entire book. Please sign up here:

New short course: Build Long-Context AI Apps with Jamba. Learn about state space models (SSMs), which have emerged as an alternative to transformers! Specifically, Jamba is a hybrid transformer-Mamba architecture that combines strengths of the transformer with ideas from SSMs. This course is built with AI21 Labs and taught by Chen Wang and Chen Almagor. The transformer architecture is computationally expensive when handling very long input contexts. But there's an alternative called Mamba, a selective state space model that can process very long contexts with a much lower computational cost. However, researchers found that the pure Mamba architecture underperforms in understanding the context, and gives lower-quality responses. To overcome this, AI21 developed the Jamba model, which combines Mamba's computational efficiency with the transformer's attention mechanism to help with the output quality. In this course, you’ll learn about how state space models, and Jamba, work. You’ll also learn how to prompt Jamba, use it to process long documents, and build long-context RAG apps. - Learn how Jamba combines transformer and state space model architectures to achieve high performance and quality - Use the AI21 SDK, with an example of prompting over a large 200k-token annual financial report of Nvidia - Use Jamba for tool-calling, with hands-on examples from calling simple arithmetic calculations to a function that returns quarterly company financial reports. - Learn how training for long context is done, and the metrics used for its evaluation - Create a RAG app using the AI21 Conversational RAG tool and build your own RAG pipeline that uses Jamba and LangChain. By the end of this course, you'll learn how to build applications that can handle context as long as an entire book. Please sign up here:

Andrew Ng

77,792 просмотров • 1 год назад

Did a very different format with Reiner Pope – a blackboard lecture where he walks through how frontier LLMs are trained and served. It's shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a bit technical, but I encourage you to hang in there - it’s really worth it. There are less than a handful of people who understand the full stack of AI, from chip design to model architecture, as well as Reiner. It was a real delight to learn from him. Recommend watching this one on YouTube so you can see the chalkboard. 0:00:00 – How batch size affects token cost and speed 0:31:59 – How MoE models are laid out across GPU racks 0:47:02 – How pipeline parallelism spreads model layers across racks 1:03:27 – Why Ilya said, “As we now know, pipelining is not wise.” 1:18:49 – Because of RL, models may be 100x over-trained beyond Chinchilla-optimal 1:32:52 – Deducing long context memory costs from API pricing 2:03:52 – Convergent evolution between neural nets and cryptography

Did a very different format with Reiner Pope – a blackboard lecture where he walks through how frontier LLMs are trained and served. It's shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a bit technical, but I encourage you to hang in there - it’s really worth it. There are less than a handful of people who understand the full stack of AI, from chip design to model architecture, as well as Reiner. It was a real delight to learn from him. Recommend watching this one on YouTube so you can see the chalkboard. 0:00:00 – How batch size affects token cost and speed 0:31:59 – How MoE models are laid out across GPU racks 0:47:02 – How pipeline parallelism spreads model layers across racks 1:03:27 – Why Ilya said, “As we now know, pipelining is not wise.” 1:18:49 – Because of RL, models may be 100x over-trained beyond Chinchilla-optimal 1:32:52 – Deducing long context memory costs from API pricing 2:03:52 – Convergent evolution between neural nets and cryptography

Dwarkesh Patel

1,291,288 просмотров • 4 месяцев назад

NVIDIA CEO Jensen Huang explains the architecture of the AI revolution AI is fundamentally about thinking—and that requires supercomputers. We’ve created a new industry: AI infrastructure. It will be as foundational as the electricity and internet infrastructures—permanent for every society. This is a transformative time; how we deliver and produce intelligence is changing. Each layer of the AI stack is growing: 1. Energy 2. Chips 3. Infrastructure 4. AI models 5. AI applications We’re witnessing a complete revolution across the stack, with opportunities at every layer. NVIDIA intends to participate across the entire stack.

NVIDIA CEO Jensen Huang explains the architecture of the AI revolution AI is fundamentally about thinking—and that requires supercomputers. We’ve created a new industry: AI infrastructure. It will be as foundational as the electricity and internet infrastructures—permanent for every society. This is a transformative time; how we deliver and produce intelligence is changing. Each layer of the AI stack is growing: 1. Energy 2. Chips 3. Infrastructure 4. AI models 5. AI applications We’re witnessing a complete revolution across the stack, with opportunities at every layer. NVIDIA intends to participate across the entire stack.

Ask Perplexity

64,013 просмотров • 10 месяцев назад

I was grateful to be joined by a few of our wonderful missionaries at the Provo Missionary Training Center recently as we discussed how missionary work—and all work in The Church of Jesus Christ of Latter-day Saints—is built on the foundation of Jesus Christ. How grateful I am to be involved in sharing the Savior’s gospel with our missionaries across the globe.

I was grateful to be joined by a few of our wonderful missionaries at the Provo Missionary Training Center recently as we discussed how missionary work—and all work in The Church of Jesus Christ of Latter-day Saints—is built on the foundation of Jesus Christ. How grateful I am to be involved in sharing the Savior’s gospel with our missionaries across the globe.

Gary E. Stevenson

30,272 просмотров • 9 месяцев назад