Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research... show more

Madison Kanna

80,045 subscribers

119,268 Aufrufe • vor 6 Monaten •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

⚡️Today marks a big milestone for Nebius. We’re launching Nebius Token Factory, the evolution of Nebius AI Studio, built to make open-source AI production-grade. Token Factory transforms raw open models into governed, scalable systems with dedicated inference, sub-second latency, 99.9% uptime and zero-retention compliance. It’s where inference, post-training and governance converge, turning raw compute into reliable intelligence. Run AI inference at scale:

⚡️Today marks a big milestone for Nebius. We’re launching Nebius Token Factory, the evolution of Nebius AI Studio, built to make open-source AI production-grade. Token Factory transforms raw open models into governed, scalable systems with dedicated inference, sub-second latency, 99.9% uptime and zero-retention compliance. It’s where inference, post-training and governance converge, turning raw compute into reliable intelligence. Run AI inference at scale:

Nebius Token Factory

260,532 Aufrufe • vor 11 Monaten

BG2. Ep 17. Double $NVDA! System Level Comp Moat, “Insane Demand”, Inference Explosion 1 B x, Memphis Supercluster, OpenAI, & more. Brad Gerstner Clark Tang Bill Gurley (00:00) Intro (1:50) The Evolution of AGI and Personal Assistants (06:03) NVIDIA's Competitive Moat (15:51 ) The Future of Inference and Training in AI (19:01) Building the AI Infrastructure (31:35) Inventing a New Market in an AI Future (38:40) The Impact of OpenAI (43:25) The Future of AI Models (46.44) and Memphis Supercluster (51:21) Distributed Computing and Inference Scaling (55:54) Inference Time Reasoning and Its Importance (01:00:46) AI's Role in Growing Business and Improving Productivity (01:08:00) Ensuring Safe AI Development (01:12:31) The Balance of Open Source and Closed Source AI

BG2. Ep 17. Double $NVDA! System Level Comp Moat, “Insane Demand”, Inference Explosion 1 B x, Memphis Supercluster, OpenAI, & more. Brad Gerstner Clark Tang Bill Gurley (00:00) Intro (1:50) The Evolution of AGI and Personal Assistants (06:03) NVIDIA's Competitive Moat (15:51 ) The Future of Inference and Training in AI (19:01) Building the AI Infrastructure (31:35) Inventing a New Market in an AI Future (38:40) The Impact of OpenAI (43:25) The Future of AI Models (46.44) and Memphis Supercluster (51:21) Distributed Computing and Inference Scaling (55:54) Inference Time Reasoning and Its Importance (01:00:46) AI's Role in Growing Business and Improving Productivity (01:08:00) Ensuring Safe AI Development (01:12:31) The Balance of Open Source and Closed Source AI

Bg2 Pod

487,379 Aufrufe • vor 1 Jahr

I sat down with Silas Alberti, Head of Research Cognition, to chat about the intersection of RL and inference -- from training frontier coding models to running inference at scale. Watch till the end for bonus doggo! :o 0:00 — Intros 1:04 — What's hardest to get right in an RL run 3:39 — How Cognition started using Modal 4:03 — What to weigh when buying inference 4:58 — The latency/throughput Pareto frontier 7:14 — DFlash reveal (!!) 8:44 — Why faster runtime matters 9:56 — Memory-bound vs. compute-bound 10:57 — Tree-based speculation 14:20 — Hot take: RL and inference optimization converging 15:28 — Online training of the speculator 16:22 — "Auto inference" 17:27 — Agents competing on optimization problems 18:27 — What inference providers still get wrong

I sat down with Silas Alberti, Head of Research Cognition, to chat about the intersection of RL and inference -- from training frontier coding models to running inference at scale. Watch till the end for bonus doggo! :o 0:00 — Intros 1:04 — What's hardest to get right in an RL run 3:39 — How Cognition started using Modal 4:03 — What to weigh when buying inference 4:58 — The latency/throughput Pareto frontier 7:14 — DFlash reveal (!!) 8:44 — Why faster runtime matters 9:56 — Memory-bound vs. compute-bound 10:57 — Tree-based speculation 14:20 — Hot take: RL and inference optimization converging 15:28 — Online training of the speculator 16:22 — "Auto inference" 17:27 — Agents competing on optimization problems 18:27 — What inference providers still get wrong

gongy

81,949 Aufrufe • vor 2 Monaten

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

Uomi

18,982 Aufrufe • vor 3 Monaten

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.

Amazon Web Services

27,021 Aufrufe • vor 7 Monaten

Jesse Pollak on the ecosystem forming around tokenized inference "There's this really cool tokenized inference market that Venice has innovated, where they've taken inference and turned it into a token called DIEM where you can buy it and you get $1 of inference per day" "That innovation of tokenizing inference has unlocked a ton of innovation around it, where people are using that tokenized inference to do things" "There's other projects that let people sell their tokenized inference at a discount so other people can get cheaper inference and the whole system works more efficiently" "I feel like that's the first time we're really seeing the intersection of AI plus markets plus crypto and it's thanks to Venice and it's happening on Base"

Jesse Pollak on the ecosystem forming around tokenized inference "There's this really cool tokenized inference market that Venice has innovated, where they've taken inference and turned it into a token called DIEM where you can buy it and you get $1 of inference per day" "That innovation of tokenizing inference has unlocked a ton of innovation around it, where people are using that tokenized inference to do things" "There's other projects that let people sell their tokenized inference at a discount so other people can get cheaper inference and the whole system works more efficiently" "I feel like that's the first time we're really seeing the intersection of AI plus markets plus crypto and it's thanks to Venice and it's happening on Base"

Market Bubble

26,240 Aufrufe • vor 2 Monaten

People assume the demand for inference will grow linearly with user adoption. However, the demand is actually growing combinatorially. Here’s why from researcher and head of training at Baseten Charlie O'Neill

People assume the demand for inference will grow linearly with user adoption. However, the demand is actually growing combinatorially. Here’s why from researcher and head of training at Baseten Charlie O'Neill

Madison Kanna

117,489 Aufrufe • vor 5 Monaten

My interview with Amazon Web Services CEO Matt Garman. 0:00 Intro 0:57 White Collar Jobs 8:51 How much of AWS's code is written by AI? 12:54 How to have a career in the AI era 15:43 AI Bottlenecks 18:06 Inference vs Training Growth 20:05 AWS Custom Silicon 25:50 Annapurna Acquisition 27:53 AI Models 33:35 Open vs Closed Models 41:28 Benchmarks 47:13 Agents

My interview with Amazon Web Services CEO Matt Garman. 0:00 Intro 0:57 White Collar Jobs 8:51 How much of AWS's code is written by AI? 12:54 How to have a career in the AI era 15:43 AI Bottlenecks 18:06 Inference vs Training Growth 20:05 AWS Custom Silicon 25:50 Annapurna Acquisition 27:53 AI Models 33:35 Open vs Closed Models 41:28 Benchmarks 47:13 Agents

Matthew Berman

49,153 Aufrufe • vor 1 Jahr

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein

a16z

87,993 Aufrufe • vor 8 Monaten

New Delphi Podcast with TravisGood, co-founder of ambient.xyz Travis believes open source AI will win and that OpenAI and Anthropic are making a major strategic mistake by betting against it We discuss – Why closed AI providers can’t be trusted as infrastructure – China’s open source advantage – What happens if OpenAI reaches AGI – Prompt injection and AI’s unsolved safety problem – The AI CapEx bubble and a possible AI winter – How Ambient is building Uber for inference - 00:00 Intro - 03:20 What Ambient Is and How It Works - 18:40 Verified AI Inference - 40:20 China and the Open-Source AI Race - 49:00 Why Closed AI Is Making a Mistake - 1:03:00 What Happens If OpenAI Reaches AGI? - 1:17:10 The AI Bubble and a Possible AI Winter

New Delphi Podcast with TravisGood, co-founder of ambient.xyz Travis believes open source AI will win and that OpenAI and Anthropic are making a major strategic mistake by betting against it We discuss – Why closed AI providers can’t be trusted as infrastructure – China’s open source advantage – What happens if OpenAI reaches AGI – Prompt injection and AI’s unsolved safety problem – The AI CapEx bubble and a possible AI winter – How Ambient is building Uber for inference - 00:00 Intro - 03:20 What Ambient Is and How It Works - 18:40 Verified AI Inference - 40:20 China and the Open-Source AI Race - 49:00 Why Closed AI Is Making a Mistake - 1:03:00 What Happens If OpenAI Reaches AGI? - 1:17:10 The AI Bubble and a Possible AI Winter

Tommy

24,586 Aufrufe • vor 2 Monaten

Most AI projects don't fail because models aren't good enough. They fail because inference economics don't scale. During #CXOSpice, Roman Chernin Nebius broke it down: ✅Closed models kill margins ✅Open models unlock control, if you can run them well ✅DataData only becomes a moat when inference is sustainable One team cut inference costs 26× in production. Only 30% of orgs use AI at scale, mostly in 1% of workloads. We're just getting started. Full conversation: #AI #Inference #TechInnovation Learn more about Nebius Token Factory:

Most AI projects don't fail because models aren't good enough. They fail because inference economics don't scale. During #CXOSpice, Roman Chernin Nebius broke it down: ✅Closed models kill margins ✅Open models unlock control, if you can run them well ✅DataData only becomes a moat when inference is sustainable One team cut inference costs 26× in production. Only 30% of orgs use AI at scale, mostly in 1% of workloads. We're just getting started. Full conversation: #AI #Inference #TechInnovation Learn more about Nebius Token Factory:

Helen Yu

13,662 Aufrufe • vor 8 Monaten

Baseten’s CEO on why every company will want to own its AI / Tuhin Srivastava on the rise of AI agents, the demand for inference, and building a new kind of hyperscaler. Lately, I’ve been spending a lot of time thinking about the AI inference market and how big it could get. Agents like Muse, Instinct, Town, and Grok Bot aggressively use browsers, run their own computers, and burn through far more tokens than a traditional chatbot. As more people put them to work (Muse is number two in the App Store), the demand for inference, or the computing needed to run these models, should grow enormously. Tuhin and I discuss the rise of agents using browsers and virtual machines to get things done, and Baseten's recent acquisition to help power that shift. We also talk about the data center backlash, why companies are embracing Chinese open models, his plans for Baseten’s new research lab, and why he thinks inference becomes the only market left after AGI. Timestamps: 00:00 What Is AI Inference? 06:05 Competing With the Cloud Giants 09:24 Why Companies Want to Own Their AI 15:30 Building Baseten Before the AI Boom 23:45 DeepSeek and the Race for Open AI Models 29:32 Baseten’s Growth and Expansion 33:05 AI Agents and the Blaxel Acquisition 38:32 Data Centers and the AI Backlash 43:25 What Happens to Inference After AGI? 45:12 When AI Agents Become Customers Thanks to the show's premier sponsors: Atlassian, Granola, and Mercury.

Baseten’s CEO on why every company will want to own its AI / Tuhin Srivastava on the rise of AI agents, the demand for inference, and building a new kind of hyperscaler. Lately, I’ve been spending a lot of time thinking about the AI inference market and how big it could get. Agents like Muse, Instinct, Town, and Grok Bot aggressively use browsers, run their own computers, and burn through far more tokens than a traditional chatbot. As more people put them to work (Muse is number two in the App Store), the demand for inference, or the computing needed to run these models, should grow enormously. Tuhin and I discuss the rise of agents using browsers and virtual machines to get things done, and Baseten's recent acquisition to help power that shift. We also talk about the data center backlash, why companies are embracing Chinese open models, his plans for Baseten’s new research lab, and why he thinks inference becomes the only market left after AGI. Timestamps: 00:00 What Is AI Inference? 06:05 Competing With the Cloud Giants 09:24 Why Companies Want to Own Their AI 15:30 Building Baseten Before the AI Boom 23:45 DeepSeek and the Race for Open AI Models 29:32 Baseten’s Growth and Expansion 33:05 AI Agents and the Blaxel Acquisition 38:32 Data Centers and the AI Backlash 43:25 What Happens to Inference After AGI? 45:12 When AI Agents Become Customers Thanks to the show's premier sponsors: Atlassian, Granola, and Mercury.

Alex Heath

26,164 Aufrufe • vor 15 Tagen

How to become an AI researcher with Charlie O'Neill Charlie co-founded Parsed to build specialized open-source models that can outperform frontier labs. I first met Charlie when Parsed was acquired by Baseten, and now he leads our model development team. Charlie is one of the smartest people I know, and I had the pleasure of talking to him about: 0:00 Intro 3:13 Leaving Oxford to start a company 6:37 Becoming an AI researcher 15:37 Developing a unique POV as your moat 22:04 Parsed origin story 26:01 Big Token, the case for open-source models 33:40 Post-training, fine-tuning, specialization 46:52 Will open models catch up with closed models? 51:50 AI-led job replacement vs job creation 54:45 How to get into inference engineering This is one of my favorite conversations I’ve had in a long time. Made with Lan behind the scenes. Enjoy!

How to become an AI researcher with Charlie O'Neill Charlie co-founded Parsed to build specialized open-source models that can outperform frontier labs. I first met Charlie when Parsed was acquired by Baseten, and now he leads our model development team. Charlie is one of the smartest people I know, and I had the pleasure of talking to him about: 0:00 Intro 3:13 Leaving Oxford to start a company 6:37 Becoming an AI researcher 15:37 Developing a unique POV as your moat 22:04 Parsed origin story 26:01 Big Token, the case for open-source models 33:40 Post-training, fine-tuning, specialization 46:52 Will open models catch up with closed models? 51:50 AI-led job replacement vs job creation 54:45 How to get into inference engineering This is one of my favorite conversations I’ve had in a long time. Made with Lan behind the scenes. Enjoy!

Madison Kanna

147,366 Aufrufe • vor 2 Monaten

Open source powers every layer of the AI software stack—from agents and models to inference, training and cloud infrastructure. At #OSSummit Korea, Linux Foundation CEO Jim Zemlin explains how open collaboration is making AI faster, more affordable and more accessible. Watch:

Open source powers every layer of the AI software stack—from agents and models to inference, training and cloud infrastructure. At #OSSummit Korea, Linux Foundation CEO Jim Zemlin explains how open collaboration is making AI faster, more affordable and more accessible. Watch:

The Linux Foundation

13,685 Aufrufe • vor 29 Tagen

Hash Rate - Ep. 155: DSperse Subnet 2: 'SSL for AI' 🧙 Guest: opendansor and HudsonGraeme Inference Labs ZK 'Proof of Inference' - Know for certain what model produced your result. 00:00 Introduction 02:06 The Evolution of SN2 Omron -> DSperse 14:48 The Origin Story of Inference Labs 22:44 Compiling Models into ZK Circuits 25:27 Optimizing Model Efficiency through Slicing 30:06 Computer Vision Applications 46:09 Early Customer Engagement and Use Cases

Hash Rate - Ep. 155: DSperse Subnet 2: 'SSL for AI' 🧙 Guest: opendansor and HudsonGraeme Inference Labs ZK 'Proof of Inference' - Know for certain what model produced your result. 00:00 Introduction 02:06 The Evolution of SN2 Omron -> DSperse 14:48 The Origin Story of Inference Labs 22:44 Compiling Models into ZK Circuits 25:27 Optimizing Model Efficiency through Slicing 30:06 Computer Vision Applications 46:09 Early Customer Engagement and Use Cases

Mark Jeffrey

15,457 Aufrufe • vor 8 Monaten

We co-led Inferact's $150M seed round to support them in their mission to build the inference engine for all current and future AI. In this episode of The Investment Memo, Lightspeed's Bucky Moore and James Alcorn sit down with Simon Mo (Co-Founder & CEO Inferact) to cover: - How vLLM grew to 60K+ GitHub stars - Why inference is shifting to the majority of compute - How vLLM evolved from a research project into the industry standard - Why building a company was the next step to push open-source inference forward 00:00 Introduction 02:03 The investment memo 04:47 Latency vs throughput vs cost 06:19 Paged attention explained 08:04 The evolution of attention 09:42 Growing the vLLM open source community 11:41 Working with hardware vendors 14:45 Deploying vLLM at large scale 16:03 Inferact's culture of openness 18:45 Building an open ecosystem and horizontal stack 19:45 Inferact's approach to fundraising 22:14 What is the future of inference? Simon Mo Bucky Moore James Alcorn

We co-led Inferact's $150M seed round to support them in their mission to build the inference engine for all current and future AI. In this episode of The Investment Memo, Lightspeed's Bucky Moore and James Alcorn sit down with Simon Mo (Co-Founder & CEO Inferact) to cover: - How vLLM grew to 60K+ GitHub stars - Why inference is shifting to the majority of compute - How vLLM evolved from a research project into the industry standard - Why building a company was the next step to push open-source inference forward 00:00 Introduction 02:03 The investment memo 04:47 Latency vs throughput vs cost 06:19 Paged attention explained 08:04 The evolution of attention 09:42 Growing the vLLM open source community 11:41 Working with hardware vendors 14:45 Deploying vLLM at large scale 16:03 Inferact's culture of openness 18:45 Building an open ecosystem and horizontal stack 19:45 Inferact's approach to fundraising 22:14 What is the future of inference? Simon Mo Bucky Moore James Alcorn

Lightspeed

27,040 Aufrufe • vor 8 Monaten

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

TBPN

33,278 Aufrufe • vor 6 Monaten

Cameron Dennis (Cameron.near): Making NEAR Protocol The Currency Of Intelligence (New Thesis) 00:00 Intro 02:08 OpenAI Creating Synthetic Training Data 04:21 Still Very Early On Private Inference 06:13 Open Source AI At 95% Cheaper Now 08:18 Staking NEAR For Private Inference TAM 10:33 Stake NEAR Get Free Inference Credits 12:44 Venice API Powers Uncensored Private AI 14:55 Private AI Has No Memory Problem 17:06 Agent That Owns Your Own Data 19:14 Large Scale Context Management Is Unsolved 21:28 KVM And CVM Explained Simply 23:51 Mark Zuckerberg Talking About CVM Now 25:52 AI Is Powered By Debt Right Now 28:05 Staking For Inference Makes AI Sustainable 30:14 Intelligence Must Be Accessible To Everyone

Cameron Dennis (Cameron.near): Making NEAR Protocol The Currency Of Intelligence (New Thesis) 00:00 Intro 02:08 OpenAI Creating Synthetic Training Data 04:21 Still Very Early On Private Inference 06:13 Open Source AI At 95% Cheaper Now 08:18 Staking NEAR For Private Inference TAM 10:33 Stake NEAR Get Free Inference Credits 12:44 Venice API Powers Uncensored Private AI 14:55 Private AI Has No Memory Problem 17:06 Agent That Owns Your Own Data 19:14 Large Scale Context Management Is Unsolved 21:28 KVM And CVM Explained Simply 23:51 Mark Zuckerberg Talking About CVM Now 25:52 AI Is Powered By Debt Right Now 28:05 Staking For Inference Makes AI Sustainable 30:14 Intelligence Must Be Accessible To Everyone

The Rollup

14,621 Aufrufe • vor 20 Tagen

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI Baseten Philip Kiely and ali explain what actually happens after a model is trained, why turning weights into a fast and reliable product creates an entirely new optimization problem, how quantization errors can cancel out to unlock more throughput, why inference teams are still finding 20–200% performance gains, how video generation runs into a quadratic attention wall, and how GLM-5.2 helped rewrite and optimize the GPU kernels serving GLM-5.2 itself.

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI Baseten Philip Kiely and ali explain what actually happens after a model is trained, why turning weights into a fast and reliable product creates an entirely new optimization problem, how quantization errors can cancel out to unlock more throughput, why inference teams are still finding 20–200% performance gains, how video generation runs into a quadratic attention wall, and how GLM-5.2 helped rewrite and optimize the GPU kernels serving GLM-5.2 itself.

Latent.Space

148,203 Aufrufe • vor 1 Monat