Loading video...

Video Failed to Load

Go Home

Distillation is the future.

mattmireles's profile picture

Matt Mireles

10,700 subscribers

24,695 views • 1 month ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

This is how gasoline is separated inside an oil distillation tower.
0:31

This is how gasoline is separated inside an oil distillation tower.

Architecture Presentation

17,801 views • 3 months ago

This is what an industrial scale distillation attack actually looks like.
0:28

This is what an industrial scale distillation attack actually looks like.

Autism Capital 🧩

133,088 views • 5 months ago

This is some serious driving! A 420-ton distillation tower, made by Iran, is being transferred
1:05

This is some serious driving! A 420-ton distillation tower, made by Iran, is being transferred

Soureh 🟩☫🟥

426,524 views • 9 months ago

The Orsknefteorgsintez refinery is burning, specifically its AVT (atmospheric-vacuum distillation) unit. This core facility processes crude oil into primary fuels.
0:23

The Orsknefteorgsintez refinery is burning, specifically its AVT (atmospheric-vacuum distillation) unit. This core facility processes crude oil into primary fuels.

NOELREPORTS 🇪🇺 🇺🇦

145,672 views • 8 months ago

Asked if China's DeepSeek stole American IP, AI Czar David Sacks says it looks like a technique called distillation was used where a student model can "suck the knowledge" out of the parent model and there is evidence that DeepSeek distilled knowledge from OpenAI's models, which is leading to efforts to develop techniques to prevent future copycats
1:10

Asked if China's DeepSeek stole American IP, AI Czar David Sacks says it looks like a technique called distillation was used where a student model can "suck the knowledge" out of the parent model and there is evidence that DeepSeek distilled knowledge from OpenAI's models, which is leading to efforts to develop techniques to prevent future copycats

Tsarathustra

710,764 views • 1 year ago

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹
1:31:19

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹

Yacine Mahdid

13,023 views • 3 months ago

Future is colorful! EFF is the future....
1:26

Future is colorful! EFF is the future....

TheeGreat Kwedini 🇿🇦

17,174 views • 2 years ago

The future of the internet is interconnected. The future is omniweb.
0:25

The future of the internet is interconnected. The future is omniweb.

Demos Network

37,431 views • 2 years ago

The future is Digital ...the future is Streaming ... Karibu 2025.
0:44

The future is Digital ...the future is Streaming ... Karibu 2025.

Kin Kariisa🇺🇬

18,322 views • 1 year ago

Our co-founder Terence Tao is announcing SAIR Foundation's inaugural competition: the Mathematics Distillation Challenge. Co-organized by Damek, Terence Tao, and SAIR Foundation.
2:12

Our co-founder Terence Tao is announcing SAIR Foundation's inaugural competition: the Mathematics Distillation Challenge. Co-organized by Damek, Terence Tao, and SAIR Foundation.

SAIR

470,211 views • 4 months ago

China's Al strategy: Ten times less spending, winning the customer race. The secret? Distillation. #Al #China
4:41

China's Al strategy: Ten times less spending, winning the customer race. The secret? Distillation. #Al #China

Johannes Maria

42,667 views • 23 days ago

data labeling is dead. long live distillation. from data to object detection endpoint in 90 seconds. link:
1:23

data labeling is dead. long live distillation. from data to object detection endpoint in 90 seconds. link:

SkalskiP

136,190 views • 7 months ago

The future is now. The future is $EMP. What's next begins here.
0:39

The future is now. The future is $EMP. What's next begins here.

Empyreal

21,138 views • 10 months ago

Distillation has paved the way for China and others to compete at the cutting edge of AI faster and cheaper than ever before Deepseek woke up the world to its disruptive potential Our latest TechCheck Take deep dive on AI's distillation problem
1:00

Distillation has paved the way for China and others to compete at the cutting edge of AI faster and cheaper than ever before Deepseek woke up the world to its disruptive potential Our latest TechCheck Take deep dive on AI's distillation problem

Deirdre Bosa

11,694 views • 1 year ago

Claude distillation has been a big topic this week while I am (coincidentally) writing Chapter 8 on model distillation. In that context, I shared some utilities to generate distillation data from all sorts of open-weight models via OpenRouter and Ollama:
1:40

Claude distillation has been a big topic this week while I am (coincidentally) writing Chapter 8 on model distillation. In that context, I shared some utilities to generate distillation data from all sorts of open-weight models via OpenRouter and Ollama:

Sebastian Raschka

62,838 views • 5 months ago

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."
2:35

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."

dnap

190,421 views • 8 days ago

AI Agents are the Future 🔮 This is the future. This is Searchpad.
0:39

AI Agents are the Future 🔮 This is the future. This is Searchpad.

Searchpad

139,012 views • 1 year ago

Chris Marker's LA JETÉE is storytelling pared all the way back, an inventive distillation of cinema that demands you lean in to the image and inspect every frame.
0:55

Chris Marker's LA JETÉE is storytelling pared all the way back, an inventive distillation of cinema that demands you lean in to the image and inspect every frame.

MUBI

16,918 views • 1 year ago

WHY $ASTER ? Decentralization is the future and ASTER IS LEADING THE FUTURE OF DEXS
1:48

WHY $ASTER ? Decentralization is the future and ASTER IS LEADING THE FUTURE OF DEXS

Julia

24,474 views • 9 months ago

The future should look like the future Robovan is 🤯
0:50

The future should look like the future Robovan is 🤯

Tesla Owners Silicon Valley

63,667,248 views • 8 months ago