Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Distillation is the future.

mattmireles's profile picture

Matt Mireles

10,700 subscribers

24,695 görüntüleme • 3 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

This is how gasoline is separated inside an oil distillation tower.
0:31

This is how gasoline is separated inside an oil distillation tower.

Architecture Presentation

17,801 görüntüleme • 5 ay önce

Delphi AI researcher Accelxr 👾 explains why China’s AI progress is more than just distillation. “Distillation can explain a large chunk of the capability and benchmark side, but it can’t explain a lot of the things they’ve shipped... The research is quite impressive.”
0:47

Delphi AI researcher Accelxr 👾 explains why China’s AI progress is more than just distillation. “Distillation can explain a large chunk of the capability and benchmark side, but it can’t explain a lot of the things they’ve shipped... The research is quite impressive.”

Delphi Digital

37,686 görüntüleme • 25 gün önce

This is what an industrial scale distillation attack actually looks like.
0:28

This is what an industrial scale distillation attack actually looks like.

Autism Capital 🧩

133,088 görüntüleme • 7 ay önce

This is some serious driving! A 420-ton distillation tower, made by Iran, is being transferred
1:05

This is some serious driving! A 420-ton distillation tower, made by Iran, is being transferred

Soureh 🟩☫🟥

426,524 görüntüleme • 11 ay önce

Asked if China's DeepSeek stole American IP, AI Czar David Sacks says it looks like a technique called distillation was used where a student model can "suck the knowledge" out of the parent model and there is evidence that DeepSeek distilled knowledge from OpenAI's models, which is leading to efforts to develop techniques to prevent future copycats
1:10

Asked if China's DeepSeek stole American IP, AI Czar David Sacks says it looks like a technique called distillation was used where a student model can "suck the knowledge" out of the parent model and there is evidence that DeepSeek distilled knowledge from OpenAI's models, which is leading to efforts to develop techniques to prevent future copycats

Tsarathustra

710,764 görüntüleme • 1 yıl önce

Future is colorful! EFF is the future....
1:26

Future is colorful! EFF is the future....

TheeGreat Kwedini 🇿🇦

17,174 görüntüleme • 2 yıl önce

The future of the internet is interconnected. The future is omniweb.
0:25

The future of the internet is interconnected. The future is omniweb.

Demos Network

37,441 görüntüleme • 2 yıl önce

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹
1:31:19

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹

Yacine Mahdid

13,109 görüntüleme • 5 ay önce

THE FAMILY IS THE FUTURE. COMMUNITY IS THE FUTURE! . . 🙏🫶
1:14

THE FAMILY IS THE FUTURE. COMMUNITY IS THE FUTURE! . . 🙏🫶

CRIS ˖ ݁𖥔 ݁˖

11,941 görüntüleme • 2 yıl önce

The future is Digital ...the future is Streaming ... Karibu 2025.
0:44

The future is Digital ...the future is Streaming ... Karibu 2025.

Kin Kariisa🇺🇬

18,322 görüntüleme • 1 yıl önce

Was really interesting to hear John, Beren, and Charlie speculate about why Sonnet 5 and Opus 5 feel like worse models than GLM 5.3 (despite the fact that Anthropic can do raw logit distillation from Fable, and can also train Sonnet/Opus on the environments from which Fable was trained). Led to some interesting thoughts about value of distillation, what it takes to do distillation effectively, and what kinds of model behaviors are hard to extract from distillation.
3:45

Was really interesting to hear John, Beren, and Charlie speculate about why Sonnet 5 and Opus 5 feel like worse models than GLM 5.3 (despite the fact that Anthropic can do raw logit distillation from Fable, and can also train Sonnet/Opus on the environments from which Fable was trained). Led to some interesting thoughts about value of distillation, what it takes to do distillation effectively, and what kinds of model behaviors are hard to extract from distillation.

Dwarkesh Patel

143,531 görüntüleme • 17 gün önce

Our co-founder Terence Tao is announcing SAIR Foundation's inaugural competition: the Mathematics Distillation Challenge. Co-organized by Damek, Terence Tao, and SAIR Foundation.
2:12

Our co-founder Terence Tao is announcing SAIR Foundation's inaugural competition: the Mathematics Distillation Challenge. Co-organized by Damek, Terence Tao, and SAIR Foundation.

SAIR

470,211 görüntüleme • 6 ay önce

China's Al strategy: Ten times less spending, winning the customer race. The secret? Distillation. #Al #China
4:41

China's Al strategy: Ten times less spending, winning the customer race. The secret? Distillation. #Al #China

Johannes Maria

44,353 görüntüleme • 2 ay önce

data labeling is dead. long live distillation. from data to object detection endpoint in 90 seconds. link:
1:23

data labeling is dead. long live distillation. from data to object detection endpoint in 90 seconds. link:

SkalskiP

136,190 görüntüleme • 9 ay önce

Distillation has paved the way for China and others to compete at the cutting edge of AI faster and cheaper than ever before Deepseek woke up the world to its disruptive potential Our latest TechCheck Take deep dive on AI's distillation problem
1:00

Distillation has paved the way for China and others to compete at the cutting edge of AI faster and cheaper than ever before Deepseek woke up the world to its disruptive potential Our latest TechCheck Take deep dive on AI's distillation problem

Deirdre Bosa

11,701 görüntüleme • 1 yıl önce

Claude distillation has been a big topic this week while I am (coincidentally) writing Chapter 8 on model distillation. In that context, I shared some utilities to generate distillation data from all sorts of open-weight models via OpenRouter and Ollama:
1:40

Claude distillation has been a big topic this week while I am (coincidentally) writing Chapter 8 on model distillation. In that context, I shared some utilities to generate distillation data from all sorts of open-weight models via OpenRouter and Ollama:

Sebastian Raschka

63,156 görüntüleme • 7 ay önce

The future is now. The future is $EMP. What's next begins here.
0:39

The future is now. The future is $EMP. What's next begins here.

Empyreal

21,138 görüntüleme • 1 yıl önce

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."
2:35

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."

dnap

191,435 görüntüleme • 2 ay önce

The future is starting to look like the future
1:28

The future is starting to look like the future

Robotaxi Radar

58,761 görüntüleme • 28 gün önce