Loading video...

Video Failed to Load

Go Home

friendly reminder: this is on Monday 🤝 → learn about speculative decoding to speed-up inference → if you're a beginner and not interested, just use that does this for you automatically 🦙

mervenoyann's profile picture

merve

93,403 subscribers

18,211 views • 13 days ago •via X (Twitter)

Science & TechnologyEducation
LIVE

Anya Rossi• Live Now

Private livecam show

3 Comments

戴卫得's profile picture
戴卫得13 days ago

The tool that handles speculative decoding automatically is the bit I'd try first. Let me feel the speedup before the lecture 😂

Adam 🤗's profile picture
Adam 🤗13 days ago

you're making me want the weekend to end faster 😔💔

Kieren Day's profile picture
Kieren Day13 days ago

The speedup lives or dies on the acceptance rate. Code and structured output are where a small draft model guesses right most often, so that's where the big wins show up. On open-ended chat, or once you're serving large batches, the gain shrinks fast.

Related Videos

While exploring diffusion language models (DLMs), we kept asking ourselves: given speculative decoding, a lossless acceleration technique, why do we still need DLMs? 🤔 Our latest work, ORL, tries to answer this question: AR optimizes intelligence 👆, speculative decoding optimizes speed 👉, and DLMs optimize the Pareto frontier of speed and intelligence 🚀. Through co-design across the algorithms, the training and inference engine, and the kernels, ORL pushes out a new Pareto frontier of intelligence and speed. Similar performance, much faster than speculative decoding. You say lossy spec? ORL-DLM is better too! 😎 🌐 Project + live demos:
1:28

While exploring diffusion language models (DLMs), we kept asking ourselves: given speculative decoding, a lossless acceleration technique, why do we still need DLMs? 🤔 Our latest work, ORL, tries to answer this question: AR optimizes intelligence 👆, speculative decoding optimizes speed 👉, and DLMs optimize the Pareto frontier of speed and intelligence 🚀. Through co-design across the algorithms, the training and inference engine, and the kernels, ORL pushes out a new Pareto frontier of intelligence and speed. Similar performance, much faster than speculative decoding. You say lossy spec? ORL-DLM is better too! 😎 🌐 Project + live demos:

Chenfeng_X

21,233 views • 15 days ago

A friendly reminder to all Speed Racer fans that this film exists
1:11

A friendly reminder to all Speed Racer fans that this film exists

Cade32X (BOILING POINT/NOT ALONE/ZAC POWER HYPE)

27,020 views • 5 months ago

This is just a friendly reminder to all states: if you receive government grants, you must follow federal laws!
2:28

This is just a friendly reminder to all states: if you receive government grants, you must follow federal laws!

Secretary Sean Duffy

185,234 views • 1 year ago

If you’re new to kettlebells, start here. This is a solid beginner workout and super easy to progress. Not just that but you can knock this one out in 30-40 minutes from home. Link in my profile for Beginner Programs. #kettlebell #workout
0:46

If you’re new to kettlebells, start here. This is a solid beginner workout and super easy to progress. Not just that but you can knock this one out in 30-40 minutes from home. Link in my profile for Beginner Programs. #kettlebell #workout

Adam Gooch

145,914 views • 10 months ago

If we want crypto to win, we have to accept that most teams will fail Unfortunately; this is needed Does that mean it’s right for teams to extract? Absolutely not But you as the user made the decision to invest in extremely speculative startups Let’s try to learn from this
1:53

If we want crypto to win, we have to accept that most teams will fail Unfortunately; this is needed Does that mean it’s right for teams to extract? Absolutely not But you as the user made the decision to invest in extremely speculative startups Let’s try to learn from this

Nill

12,921 views • 4 months ago

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio
3:46

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

0xSero

21,768 views • 5 months ago

Welp, it happened again. This is yet another reminder that if you miss your exit or you're confused about which exit to take, do not stop on the freeway. We don't know why this driver did this, but it resulted in a crash and several close calls - and they just drove off.
1:02

Welp, it happened again. This is yet another reminder that if you miss your exit or you're confused about which exit to take, do not stop on the freeway. We don't know why this driver did this, but it resulted in a crash and several close calls - and they just drove off.

Matt Bruning

285,867 views • 2 months ago

If you support WWIII PLEASE listen to this first. This woman is spot on. It's going to cause mental hardships for a lot of you. Just know that you're not alone.
4:10

If you support WWIII PLEASE listen to this first. This woman is spot on. It's going to cause mental hardships for a lot of you. Just know that you're not alone.

The Disrespected Trucker

222,467 views • 2 years ago

Luminal fuses entire models into a single GPU kernel, automatically. Let's talk about why this matters for inference at the speed of light:
9:24

Luminal fuses entire models into a single GPU kernel, automatically. Let's talk about why this matters for inference at the speed of light:

Joe Fioti

24,436 views • 9 months ago

This coming Sunday (April 7th), I'm planning on streaming a beginner animation implementation demo for Unreal! If you're an animator without much engine experience and want to learn, I'd like to help! I'll try to go over setting up this animation blueprint from scratch.
1:28

This coming Sunday (April 7th), I'm planning on streaming a beginner animation implementation demo for Unreal! If you're an animator without much engine experience and want to learn, I'd like to help! I'll try to go over setting up this animation blueprint from scratch.

@izzylizard.bsky.social

30,718 views • 2 years ago

Neil on why AI spending today is different from the dot-com bubble: "A lot of the investment in networking equipment historically was speculative. We anticipated this future demand for users that never came. What's interesting about token consumption is that it's no longer speculative. People buy tokens because they're immediately valuable to them. You don't hoard tokens. You use them immediately. This is even different from what we had two years ago, where there was a supply crunch for Hopper generation chips in 2023, 2024. In that period it was all training oriented spend, and training is inherently speculative. I do think inference spend monotonically increases. There's no speculation on inference spend."
0:59

Neil on why AI spending today is different from the dot-com bubble: "A lot of the investment in networking equipment historically was speculative. We anticipated this future demand for users that never came. What's interesting about token consumption is that it's no longer speculative. People buy tokens because they're immediately valuable to them. You don't hoard tokens. You use them immediately. This is even different from what we had two years ago, where there was a supply crunch for Hopper generation chips in 2023, 2024. In that period it was all training oriented spend, and training is inherently speculative. I do think inference spend monotonically increases. There's no speculation on inference spend."

Patrick OShaughnessy

59,906 views • 2 months ago

Just a friendly reminder that this same ballpark was completely filled up not even 3 years ago 💔💔💔
0:59

Just a friendly reminder that this same ballpark was completely filled up not even 3 years ago 💔💔💔

PHILLIES WILL WIN 2026 WORLD SERIES 🇺🇸

393,556 views • 6 months ago

So you asked, here a little debunk about Threat Interactive. I will not lose anymore my time on this, I don't want to start a drama or a war, so if there is an answer, i will not responde. I'm not interested in, i just want people to stop use those video as a proof when this is not, cause this is really annoying for us
22:02

So you asked, here a little debunk about Threat Interactive. I will not lose anymore my time on this, I don't want to start a drama or a war, so if there is an answer, i will not responde. I'm not interested in, i just want people to stop use those video as a proof when this is not, cause this is really annoying for us

Aherys

49,426 views • 1 year ago

If you only want to hear about Jewish influence and never Russian influence, Nick Fuentes isn't for you “You're not interested in thinking, you're not interested in looking at the facts, you're interested in hearing me preach to the choir.”
5:01

If you only want to hear about Jewish influence and never Russian influence, Nick Fuentes isn't for you “You're not interested in thinking, you're not interested in looking at the facts, you're interested in hearing me preach to the choir.”

Fuentes Updates

79,517 views • 3 months ago

If you're serious about mastering Claude, then this is the most important video you'll ever watch. How to go from Beginner → Pro in 3 steps, with the exact memory system I use to set up Cowork + Claude Code for the best results. This video will permanently change how you use Claude:
20:59

If you're serious about mastering Claude, then this is the most important video you'll ever watch. How to go from Beginner → Pro in 3 steps, with the exact memory system I use to set up Cowork + Claude Code for the best results. This video will permanently change how you use Claude:

Miles Deutscher

138,533 views • 6 months ago