正在加载视频...

视频加载失败

返回首页

friendly reminder: this is on Monday 🤝 → learn about speculative decoding to speed-up inference → if you're a beginner and not interested, just use that does this for you automatically 🦙

mervenoyann's profile picture

merve

93,403 subscribers

18,211 次观看 • 15 天前 •via X (Twitter)

科学技术教育
LIVE

Anya Rossi• Live Now

Private livecam show

3 条评论

戴卫得 的头像
戴卫得14 天前

The tool that handles speculative decoding automatically is the bit I'd try first. Let me feel the speedup before the lecture 😂

Adam 🤗 的头像
Adam 🤗15 天前

you're making me want the weekend to end faster 😔💔

Kieren Day 的头像
Kieren Day15 天前

The speedup lives or dies on the acceptance rate. Code and structured output are where a small draft model guesses right most often, so that's where the big wins show up. On open-ended chat, or once you're serving large batches, the gain shrinks fast.

相关视频

While exploring diffusion language models (DLMs), we kept asking ourselves: given speculative decoding, a lossless acceleration technique, why do we still need DLMs? 🤔 Our latest work, ORL, tries to answer this question: AR optimizes intelligence 👆, speculative decoding optimizes speed 👉, and DLMs optimize the Pareto frontier of speed and intelligence 🚀. Through co-design across the algorithms, the training and inference engine, and the kernels, ORL pushes out a new Pareto frontier of intelligence and speed. Similar performance, much faster than speculative decoding. You say lossy spec? ORL-DLM is better too! 😎 🌐 Project + live demos:
1:28

While exploring diffusion language models (DLMs), we kept asking ourselves: given speculative decoding, a lossless acceleration technique, why do we still need DLMs? 🤔 Our latest work, ORL, tries to answer this question: AR optimizes intelligence 👆, speculative decoding optimizes speed 👉, and DLMs optimize the Pareto frontier of speed and intelligence 🚀. Through co-design across the algorithms, the training and inference engine, and the kernels, ORL pushes out a new Pareto frontier of intelligence and speed. Similar performance, much faster than speculative decoding. You say lossy spec? ORL-DLM is better too! 😎 🌐 Project + live demos:

Chenfeng_X

21,240 次观看 • 16 天前

A friendly reminder to all Speed Racer fans that this film exists
1:11

A friendly reminder to all Speed Racer fans that this film exists

Cade32X (BOILING POINT/NOT ALONE/ZAC POWER HYPE)

27,020 次观看 • 6 个月前

This is just a friendly reminder to all states: if you receive government grants, you must follow federal laws!
2:28

This is just a friendly reminder to all states: if you receive government grants, you must follow federal laws!

Secretary Sean Duffy

185,234 次观看 • 1 年前

If you’re new to kettlebells, start here. This is a solid beginner workout and super easy to progress. Not just that but you can knock this one out in 30-40 minutes from home. Link in my profile for Beginner Programs. #kettlebell #workout
0:46

If you’re new to kettlebells, start here. This is a solid beginner workout and super easy to progress. Not just that but you can knock this one out in 30-40 minutes from home. Link in my profile for Beginner Programs. #kettlebell #workout

Adam Gooch

145,914 次观看 • 10 个月前

If we want crypto to win, we have to accept that most teams will fail Unfortunately; this is needed Does that mean it’s right for teams to extract? Absolutely not But you as the user made the decision to invest in extremely speculative startups Let’s try to learn from this
1:53

If we want crypto to win, we have to accept that most teams will fail Unfortunately; this is needed Does that mean it’s right for teams to extract? Absolutely not But you as the user made the decision to invest in extremely speculative startups Let’s try to learn from this

Nill

12,921 次观看 • 4 个月前

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio
3:46

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

0xSero

21,768 次观看 • 5 个月前

Welp, it happened again. This is yet another reminder that if you miss your exit or you're confused about which exit to take, do not stop on the freeway. We don't know why this driver did this, but it resulted in a crash and several close calls - and they just drove off.
1:02

Welp, it happened again. This is yet another reminder that if you miss your exit or you're confused about which exit to take, do not stop on the freeway. We don't know why this driver did this, but it resulted in a crash and several close calls - and they just drove off.

Matt Bruning

285,867 次观看 • 2 个月前

If you support WWIII PLEASE listen to this first. This woman is spot on. It's going to cause mental hardships for a lot of you. Just know that you're not alone.
4:10

If you support WWIII PLEASE listen to this first. This woman is spot on. It's going to cause mental hardships for a lot of you. Just know that you're not alone.

The Disrespected Trucker

222,467 次观看 • 2 年前

Luminal fuses entire models into a single GPU kernel, automatically. Let's talk about why this matters for inference at the speed of light:
9:24

Luminal fuses entire models into a single GPU kernel, automatically. Let's talk about why this matters for inference at the speed of light:

Joe Fioti

24,436 次观看 • 9 个月前

This coming Sunday (April 7th), I'm planning on streaming a beginner animation implementation demo for Unreal! If you're an animator without much engine experience and want to learn, I'd like to help! I'll try to go over setting up this animation blueprint from scratch.
1:28

This coming Sunday (April 7th), I'm planning on streaming a beginner animation implementation demo for Unreal! If you're an animator without much engine experience and want to learn, I'd like to help! I'll try to go over setting up this animation blueprint from scratch.

@izzylizard.bsky.social

30,718 次观看 • 2 年前

Neil on why AI spending today is different from the dot-com bubble: "A lot of the investment in networking equipment historically was speculative. We anticipated this future demand for users that never came. What's interesting about token consumption is that it's no longer speculative. People buy tokens because they're immediately valuable to them. You don't hoard tokens. You use them immediately. This is even different from what we had two years ago, where there was a supply crunch for Hopper generation chips in 2023, 2024. In that period it was all training oriented spend, and training is inherently speculative. I do think inference spend monotonically increases. There's no speculation on inference spend."
0:59

Neil on why AI spending today is different from the dot-com bubble: "A lot of the investment in networking equipment historically was speculative. We anticipated this future demand for users that never came. What's interesting about token consumption is that it's no longer speculative. People buy tokens because they're immediately valuable to them. You don't hoard tokens. You use them immediately. This is even different from what we had two years ago, where there was a supply crunch for Hopper generation chips in 2023, 2024. In that period it was all training oriented spend, and training is inherently speculative. I do think inference spend monotonically increases. There's no speculation on inference spend."

Patrick OShaughnessy

59,906 次观看 • 2 个月前

Just a friendly reminder that this same ballpark was completely filled up not even 3 years ago 💔💔💔
0:59

Just a friendly reminder that this same ballpark was completely filled up not even 3 years ago 💔💔💔

PHILLIES WILL WIN 2026 WORLD SERIES 🇺🇸

393,556 次观看 • 6 个月前

So you asked, here a little debunk about Threat Interactive. I will not lose anymore my time on this, I don't want to start a drama or a war, so if there is an answer, i will not responde. I'm not interested in, i just want people to stop use those video as a proof when this is not, cause this is really annoying for us
22:02

So you asked, here a little debunk about Threat Interactive. I will not lose anymore my time on this, I don't want to start a drama or a war, so if there is an answer, i will not responde. I'm not interested in, i just want people to stop use those video as a proof when this is not, cause this is really annoying for us

Aherys

49,426 次观看 • 1 年前

If you only want to hear about Jewish influence and never Russian influence, Nick Fuentes isn't for you “You're not interested in thinking, you're not interested in looking at the facts, you're interested in hearing me preach to the choir.”
5:01

If you only want to hear about Jewish influence and never Russian influence, Nick Fuentes isn't for you “You're not interested in thinking, you're not interested in looking at the facts, you're interested in hearing me preach to the choir.”

Fuentes Updates

79,517 次观看 • 3 个月前

If you're serious about mastering Claude, then this is the most important video you'll ever watch. How to go from Beginner → Pro in 3 steps, with the exact memory system I use to set up Cowork + Claude Code for the best results. This video will permanently change how you use Claude:
20:59

If you're serious about mastering Claude, then this is the most important video you'll ever watch. How to go from Beginner → Pro in 3 steps, with the exact memory system I use to set up Cowork + Claude Code for the best results. This video will permanently change how you use Claude:

Miles Deutscher

138,533 次观看 • 6 个月前