Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Here's my conversation with Philip Kiely. A deep dive into inference engineering, his book his available online for free, and it's worth reading. Inference is becoming more important to people and organisations by the day, learning about the topic will prepare you well.

0xSero's profile picture

0xSero

60,838 subscribers

29,537 görüntüleme • 1 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering
27:58

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

Madison Kanna

119,268 görüntüleme • 5 ay önce

Jesse Pollak on the ecosystem forming around tokenized inference "There's this really cool tokenized inference market that Venice has innovated, where they've taken inference and turned it into a token called DIEM where you can buy it and you get $1 of inference per day" "That innovation of tokenizing inference has unlocked a ton of innovation around it, where people are using that tokenized inference to do things" "There's other projects that let people sell their tokenized inference at a discount so other people can get cheaper inference and the whole system works more efficiently" "I feel like that's the first time we're really seeing the intersection of AI plus markets plus crypto and it's thanks to Venice and it's happening on Base"
1:13

Jesse Pollak on the ecosystem forming around tokenized inference "There's this really cool tokenized inference market that Venice has innovated, where they've taken inference and turned it into a token called DIEM where you can buy it and you get $1 of inference per day" "That innovation of tokenizing inference has unlocked a ton of innovation around it, where people are using that tokenized inference to do things" "There's other projects that let people sell their tokenized inference at a discount so other people can get cheaper inference and the whole system works more efficiently" "I feel like that's the first time we're really seeing the intersection of AI plus markets plus crypto and it's thanks to Venice and it's happening on Base"

Market Bubble

26,240 görüntüleme • 1 ay önce

The Rose Field by Philip Pullman is out today: the conclusion to The Book of Dust and His Dark Materials. I narrated the audiobook, and Philip and I had a conversation about the novels and their legacy, which you can hear in full at the end of it:
0:59

The Rose Field by Philip Pullman is out today: the conclusion to The Book of Dust and His Dark Materials. I narrated the audiobook, and Philip and I had a conversation about the novels and their legacy, which you can hear in full at the end of it:

michael sheen 💙

37,696 görüntüleme • 10 ay önce

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI Baseten Philip Kiely and ali explain what actually happens after a model is trained, why turning weights into a fast and reliable product creates an entirely new optimization problem, how quantization errors can cancel out to unlock more throughput, why inference teams are still finding 20–200% performance gains, how video generation runs into a quadratic attention wall, and how GLM-5.2 helped rewrite and optimize the GPU kernels serving GLM-5.2 itself.
0:39

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI Baseten Philip Kiely and ali explain what actually happens after a model is trained, why turning weights into a fast and reliable product creates an entirely new optimization problem, how quantization errors can cancel out to unlock more throughput, why inference teams are still finding 20–200% performance gains, how video generation runs into a quadratic attention wall, and how GLM-5.2 helped rewrite and optimize the GPU kernels serving GLM-5.2 itself.

Latent.Space

142,961 görüntüleme • 22 gün önce

Groq founder Jonathan Ross explains why their architecture is better for inference than GPUs. Inference doesn't require as much compute power as training. What matters in inference is speed, cost, and energy consumption. $NVDA was going to lose market share as AI workloads shift to inference. They saw this and made a highly strategic move by striking a deal with Groq. $NVDA is already dominating training, and now it's also well-positioned to dominate inference.
1:45

Groq founder Jonathan Ross explains why their architecture is better for inference than GPUs. Inference doesn't require as much compute power as training. What matters in inference is speed, cost, and energy consumption. $NVDA was going to lose market share as AI workloads shift to inference. They saw this and made a highly strategic move by striking a deal with Groq. $NVDA is already dominating training, and now it's also well-positioned to dominate inference.

Oguz Erkan

97,186 görüntüleme • 8 ay önce

ryeowook learning to play basketball by reading a book about it, so he could play and get closer with his hyungs will always be one of the cutest and most precious stories i've ever heard about him
0:38

ryeowook learning to play basketball by reading a book about it, so he could play and get closer with his hyungs will always be one of the cutest and most precious stories i've ever heard about him

mint

26,367 görüntüleme • 1 yıl önce

$SNDK CEO David Goeckeler says inference is a memory-bound problem and HBF could break through some bottlenecks, so it's an enormous opportunity as inference scales, which they'll talk more at their analyst day next week. "Look, there's a ton of innovation going on right now in inference memory architectures, which we think is fantastic, very healthy." "There's a lot of different ideas. We're going to dive into this a little deeper next week at our analyst day…" "But yes, all these are opportunities for us. We think especially inference is a memory-bound problem. Storage is extraordinarily important to the equation." "We showed some stuff in our FMS keynote just a couple of hours ago about how, when you use HBF, we simulate performance and maybe break through some bottlenecks…" "As we scale inference, we think it provides an enormous opportunity. The way we're thinking about this is staying very close to our customers, because they're going to be the ones that define what the architecture is in the future."
0:57

$SNDK CEO David Goeckeler says inference is a memory-bound problem and HBF could break through some bottlenecks, so it's an enormous opportunity as inference scales, which they'll talk more at their analyst day next week. "Look, there's a ton of innovation going on right now in inference memory architectures, which we think is fantastic, very healthy." "There's a lot of different ideas. We're going to dive into this a little deeper next week at our analyst day…" "But yes, all these are opportunities for us. We think especially inference is a memory-bound problem. Storage is extraordinarily important to the equation." "We showed some stuff in our FMS keynote just a couple of hours ago about how, when you use HBF, we simulate performance and maybe break through some bottlenecks…" "As we scale inference, we think it provides an enormous opportunity. The way we're thinking about this is staying very close to our customers, because they're going to be the ones that define what the architecture is in the future."

Fireside Alpha

16,326 görüntüleme • 20 gün önce

t’s book season…..my book UNCOMMON FAVOR is ready for your learning reading, leisure reading, book club reading, audio listening and all the convenience of just hanging out with you and my book. Super excited to take you on this journey! Link in the bio! #BlackPrivilegePublishing Charlamagne Tha God Atria Books @SimonandSchuster
0:56

t’s book season…..my book UNCOMMON FAVOR is ready for your learning reading, leisure reading, book club reading, audio listening and all the convenience of just hanging out with you and my book. Super excited to take you on this journey! Link in the bio! #BlackPrivilegePublishing Charlamagne Tha God Atria Books @SimonandSchuster

dawnstaley

142,565 görüntüleme • 1 yıl önce

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”
0:52

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”

TBPN

17,057 görüntüleme • 11 ay önce

This is why Nvidia bought Groq. If Groq deploys a ton of low-cost inference chips, demand for training will increase massively. The more inference you have, the more training you need, and vice versa. Nvidia can keep its high margins selling chips for training forever.
0:35

This is why Nvidia bought Groq. If Groq deploys a ton of low-cost inference chips, demand for training will increase massively. The more inference you have, the more training you need, and vice versa. Nvidia can keep its high margins selling chips for training forever.

Lin

92,571 görüntüleme • 8 ay önce

Exciting news: I've written a book – and it's available for pre-order today! Visit to pre-order. “The Other Side of Change” is a deep dive into how we find meaning and opportunity in the tumult of change.
2:42

Exciting news: I've written a book – and it's available for pre-order today! Visit to pre-order. “The Other Side of Change” is a deep dive into how we find meaning and opportunity in the tumult of change.

Maya Shankar @ A Slight Change of Plans

26,633 görüntüleme • 1 yıl önce

18 teens with a mission: to help shape a positive online ecosystem for future generations. See how our inaugural Council for Digital Well-Being tackled tough online safety topics and advanced an important conversation about digital-well being. Read more: and watch the full video:
0:44

18 teens with a mission: to help shape a positive online ecosystem for future generations. See how our inaugural Council for Digital Well-Being tackled tough online safety topics and advanced an important conversation about digital-well being. Read more: and watch the full video:

Snap Inc.

11,672 görüntüleme • 10 ay önce

The #NLCS matchup is set! #MLBTonight takes a deep dive into the Brewers and Dodgers as they prepare to battle for the pennant!
4:45

The #NLCS matchup is set! #MLBTonight takes a deep dive into the Brewers and Dodgers as they prepare to battle for the pennant!

MLB Network

45,049 görüntüleme • 10 ay önce

Henry Shevlin and I were thrilled to be joined by Anil Seth for the latest episode of our AI podcast. The conversation was a deep dive into Anil's very interesting views about consciousness, including his scepticism about the possibility of AI consciousness. (1/2)
1:57

Henry Shevlin and I were thrilled to be joined by Anil Seth for the latest episode of our AI podcast. The conversation was a deep dive into Anil's very interesting views about consciousness, including his scepticism about the possibility of AI consciousness. (1/2)

Dan Williams

13,924 görüntüleme • 6 ay önce

I have no idea where this video is from. I found it on Reddit, but this explains humanity so well. This man walks into a Chinese Restaurant for takeout. He immediately begins speaking perfect Chinese, with a perfect accent. He has a conversation with the staff, who calls him "one of us," and then they proceed to give him his food for free. This is humanity. Everyone should take a chance to learn about others, identify with others, and empathize with others. We often see more racist people in small towns rather than in large cities. This isn't necessarily their faults. This is because racism is mostly built on "the unknown" and people tend to fear the unknown. Once you get to know people and befriend people from other races, religions, sexual orientations, gender ideologies, and ethnicities, you have a tendency to better understand where they come from. This is why learning about other cultures, religions, and even languages is so important. It's why studying history, and sociology are important. Reach out to someone you don't identify well with and try to learn about them today. It will help humanity tomorrow.
1:24

I have no idea where this video is from. I found it on Reddit, but this explains humanity so well. This man walks into a Chinese Restaurant for takeout. He immediately begins speaking perfect Chinese, with a perfect accent. He has a conversation with the staff, who calls him "one of us," and then they proceed to give him his food for free. This is humanity. Everyone should take a chance to learn about others, identify with others, and empathize with others. We often see more racist people in small towns rather than in large cities. This isn't necessarily their faults. This is because racism is mostly built on "the unknown" and people tend to fear the unknown. Once you get to know people and befriend people from other races, religions, sexual orientations, gender ideologies, and ethnicities, you have a tendency to better understand where they come from. This is why learning about other cultures, religions, and even languages is so important. It's why studying history, and sociology are important. Reach out to someone you don't identify well with and try to learn about them today. It will help humanity tomorrow.

Ed Krassenstein

5,802,416 görüntüleme • 2 yıl önce

It's time to familiarize yourself with the Energy Web Ecosystem! Prepare for the Energy Web Ecosystem DEEP DIVE series, where we delve into the application suite and solutions, all powered by $EWT 🔵🟣
0:42

It's time to familiarize yourself with the Energy Web Ecosystem! Prepare for the Energy Web Ecosystem DEEP DIVE series, where we delve into the application suite and solutions, all powered by $EWT 🔵🟣

Energy Web

31,545 görüntüleme • 2 yıl önce