正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research... show more

Madison Kanna

80,045 subscribers

119,268 次观看 • 3 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

⚡️Today marks a big milestone for Nebius. We’re launching Nebius Token Factory, the evolution of Nebius AI Studio, built to make open-source AI production-grade. Token Factory transforms raw open models into governed, scalable systems with dedicated inference, sub-second latency, 99.9% uptime and zero-retention compliance. It’s where inference, post-training and governance converge, turning raw compute into reliable intelligence. Run AI inference at scale:

⚡️Today marks a big milestone for Nebius. We’re launching Nebius Token Factory, the evolution of Nebius AI Studio, built to make open-source AI production-grade. Token Factory transforms raw open models into governed, scalable systems with dedicated inference, sub-second latency, 99.9% uptime and zero-retention compliance. It’s where inference, post-training and governance converge, turning raw compute into reliable intelligence. Run AI inference at scale:

Nebius Token Factory

260,215 次观看 • 8 个月前

BG2. Ep 17. Double $NVDA! System Level Comp Moat, “Insane Demand”, Inference Explosion 1 B x, Memphis Supercluster, OpenAI, & more. Brad Gerstner Clark Tang Bill Gurley (00:00) Intro (1:50) The Evolution of AGI and Personal Assistants (06:03) NVIDIA's Competitive Moat (15:51 ) The Future of Inference and Training in AI (19:01) Building the AI Infrastructure (31:35) Inventing a New Market in an AI Future (38:40) The Impact of OpenAI (43:25) The Future of AI Models (46.44) and Memphis Supercluster (51:21) Distributed Computing and Inference Scaling (55:54) Inference Time Reasoning and Its Importance (01:00:46) AI's Role in Growing Business and Improving Productivity (01:08:00) Ensuring Safe AI Development (01:12:31) The Balance of Open Source and Closed Source AI

BG2. Ep 17. Double $NVDA! System Level Comp Moat, “Insane Demand”, Inference Explosion 1 B x, Memphis Supercluster, OpenAI, & more. Brad Gerstner Clark Tang Bill Gurley (00:00) Intro (1:50) The Evolution of AGI and Personal Assistants (06:03) NVIDIA's Competitive Moat (15:51 ) The Future of Inference and Training in AI (19:01) Building the AI Infrastructure (31:35) Inventing a New Market in an AI Future (38:40) The Impact of OpenAI (43:25) The Future of AI Models (46.44) and Memphis Supercluster (51:21) Distributed Computing and Inference Scaling (55:54) Inference Time Reasoning and Its Importance (01:00:46) AI's Role in Growing Business and Improving Productivity (01:08:00) Ensuring Safe AI Development (01:12:31) The Balance of Open Source and Closed Source AI

Bg2 Pod

487,379 次观看 • 1 年前

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

Uomi

18,816 次观看 • 19 天前

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”

TBPN

17,057 次观看 • 10 个月前

"The inflection point for inference has arrived." — Jensen Huang, Founder & CEO of NVIDIA We’ve officially crossed a new milestone in the inference era — where widespread adoption of AI shifts from learning to doing. The breakthrough: extreme codesign across hardware and software driving down cost per token. Lower cost → more inference More inference → more users and applications More users and applications → exponential AI revenue growth

"The inflection point for inference has arrived." — Jensen Huang, Founder & CEO of NVIDIA We’ve officially crossed a new milestone in the inference era — where widespread adoption of AI shifts from learning to doing. The breakthrough: extreme codesign across hardware and software driving down cost per token. Lower cost → more inference More inference → more users and applications More users and applications → exponential AI revenue growth

NVIDIA

81,834 次观看 • 3 个月前

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.

Amazon Web Services

27,021 次观看 • 4 个月前

Here's my conversation with Philip Kiely. A deep dive into inference engineering, his book his available online for free, and it's worth reading. Inference is becoming more important to people and organisations by the day, learning about the topic will prepare you well.

Here's my conversation with Philip Kiely. A deep dive into inference engineering, his book his available online for free, and it's worth reading. Inference is becoming more important to people and organisations by the day, learning about the topic will prepare you well.

0xSero

24,290 次观看 • 8 天前

inference engineering everyone 💚

inference engineering everyone 💚

Kate Deyneka

72,838 次观看 • 3 个月前

Most AI projects don't fail because models aren't good enough. They fail because inference economics don't scale. During #CXOSpice, Roman Chernin Nebius broke it down: ✅Closed models kill margins ✅Open models unlock control, if you can run them well ✅DataData only becomes a moat when inference is sustainable One team cut inference costs 26× in production. Only 30% of orgs use AI at scale, mostly in 1% of workloads. We're just getting started. Full conversation: #AI #Inference #TechInnovation Learn more about Nebius Token Factory:

Most AI projects don't fail because models aren't good enough. They fail because inference economics don't scale. During #CXOSpice, Roman Chernin Nebius broke it down: ✅Closed models kill margins ✅Open models unlock control, if you can run them well ✅DataData only becomes a moat when inference is sustainable One team cut inference costs 26× in production. Only 30% of orgs use AI at scale, mostly in 1% of workloads. We're just getting started. Full conversation: #AI #Inference #TechInnovation Learn more about Nebius Token Factory:

Helen Yu

13,662 次观看 • 5 个月前

My interview with Amazon Web Services CEO Matt Garman. 0:00 Intro 0:57 White Collar Jobs 8:51 How much of AWS's code is written by AI? 12:54 How to have a career in the AI era 15:43 AI Bottlenecks 18:06 Inference vs Training Growth 20:05 AWS Custom Silicon 25:50 Annapurna Acquisition 27:53 AI Models 33:35 Open vs Closed Models 41:28 Benchmarks 47:13 Agents

My interview with Amazon Web Services CEO Matt Garman. 0:00 Intro 0:57 White Collar Jobs 8:51 How much of AWS's code is written by AI? 12:54 How to have a career in the AI era 15:43 AI Bottlenecks 18:06 Inference vs Training Growth 20:05 AWS Custom Silicon 25:50 Annapurna Acquisition 27:53 AI Models 33:35 Open vs Closed Models 41:28 Benchmarks 47:13 Agents

Matthew Berman

49,126 次观看 • 10 个月前

Inference Engineering launches today.

Inference Engineering launches today.

Philip Kiely

1,386,678 次观看 • 4 个月前

People assume the demand for inference will grow linearly with user adoption. However, the demand is actually growing combinatorially. Here’s why from researcher and head of training at Baseten Charlie O'Neill

People assume the demand for inference will grow linearly with user adoption. However, the demand is actually growing combinatorially. Here’s why from researcher and head of training at Baseten Charlie O'Neill

Madison Kanna

117,489 次观看 • 3 个月前

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein

a16z

87,993 次观看 • 5 个月前

will open-source models become the majority of inference? dax weighs in: YES

will open-source models become the majority of inference? dax weighs in: YES

Lan

29,514 次观看 • 4 天前

Hash Rate - Ep. 155: DSperse Subnet 2: 'SSL for AI' 🧙 Guest: opendansor and HudsonGraeme Inference Labs ZK 'Proof of Inference' - Know for certain what model produced your result. 00:00 Introduction 02:06 The Evolution of SN2 Omron -> DSperse 14:48 The Origin Story of Inference Labs 22:44 Compiling Models into ZK Circuits 25:27 Optimizing Model Efficiency through Slicing 30:06 Computer Vision Applications 46:09 Early Customer Engagement and Use Cases

Hash Rate - Ep. 155: DSperse Subnet 2: 'SSL for AI' 🧙 Guest: opendansor and HudsonGraeme Inference Labs ZK 'Proof of Inference' - Know for certain what model produced your result. 00:00 Introduction 02:06 The Evolution of SN2 Omron -> DSperse 14:48 The Origin Story of Inference Labs 22:44 Compiling Models into ZK Circuits 25:27 Optimizing Model Efficiency through Slicing 30:06 Computer Vision Applications 46:09 Early Customer Engagement and Use Cases

Mark Jeffrey

15,221 次观看 • 5 个月前

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

TBPN

33,278 次观看 • 3 个月前

Ready to experience the future of #AI? #Inferium's MVP is coming. Be among the first to: 🧠 Evaluate cutting-edge AI models ⚡ Run #inference on AI models 💰 Earn rewards & top the leaderboard Join waitlist now to secure early access and exclusive perks: #Inferium #MVP #AI #Inference

Ready to experience the future of #AI? #Inferium's MVP is coming. Be among the first to: 🧠 Evaluate cutting-edge AI models ⚡ Run #inference on AI models 💰 Earn rewards & top the leaderboard Join waitlist now to secure early access and exclusive perks: #Inferium #MVP #AI #Inference

Inferium

37,316 次观看 • 1 年前

Open source like DeepSeek doesn’t shrink markets. It grows them. Faster, cheaper training means rising inference demand and a new era of AI hardware.

Open source like DeepSeek doesn’t shrink markets. It grows them. Faster, cheaper training means rising inference demand and a new era of AI hardware.

The Linux Foundation

32,930 次观看 • 1 年前

We co-led Inferact's $150M seed round to support them in their mission to build the inference engine for all current and future AI. In this episode of The Investment Memo, Lightspeed's Bucky Moore and James Alcorn sit down with Simon Mo (Co-Founder & CEO Inferact) to cover: - How vLLM grew to 60K+ GitHub stars - Why inference is shifting to the majority of compute - How vLLM evolved from a research project into the industry standard - Why building a company was the next step to push open-source inference forward 00:00 Introduction 02:03 The investment memo 04:47 Latency vs throughput vs cost 06:19 Paged attention explained 08:04 The evolution of attention 09:42 Growing the vLLM open source community 11:41 Working with hardware vendors 14:45 Deploying vLLM at large scale 16:03 Inferact's culture of openness 18:45 Building an open ecosystem and horizontal stack 19:45 Inferact's approach to fundraising 22:14 What is the future of inference? Simon Mo Bucky Moore James Alcorn

We co-led Inferact's $150M seed round to support them in their mission to build the inference engine for all current and future AI. In this episode of The Investment Memo, Lightspeed's Bucky Moore and James Alcorn sit down with Simon Mo (Co-Founder & CEO Inferact) to cover: - How vLLM grew to 60K+ GitHub stars - Why inference is shifting to the majority of compute - How vLLM evolved from a research project into the industry standard - Why building a company was the next step to push open-source inference forward 00:00 Introduction 02:03 The investment memo 04:47 Latency vs throughput vs cost 06:19 Paged attention explained 08:04 The evolution of attention 09:42 Growing the vLLM open source community 11:41 Working with hardware vendors 14:45 Deploying vLLM at large scale 16:03 Inferact's culture of openness 18:45 Building an open ecosystem and horizontal stack 19:45 Inferact's approach to fundraising 22:14 What is the future of inference? Simon Mo Bucky Moore James Alcorn

Lightspeed

27,040 次观看 • 5 个月前

Thank you to Markku Räsänen, CEO of ConfidentialMind for sitting down with us and sharing his thoughts on: - Whether open source models are undervalued - Changes in AI infrastructure & inference - Power consumption & efficiency in Europe - The future of open source AI

Thank you to Markku Räsänen, CEO of ConfidentialMind for sitting down with us and sharing his thoughts on: - Whether open source models are undervalued - Changes in AI infrastructure & inference - Power consumption & efficiency in Europe - The future of open source AI

SambaNova

31,873 次观看 • 11 个月前