正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Distillation is the future.

Matt Mireles

10,700 subscribers

24,695 次观看 • 1 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

This is how gasoline is separated inside an oil distillation tower.

This is how gasoline is separated inside an oil distillation tower.

Architecture Presentation

17,801 次观看 • 3 个月前

This is what an industrial scale distillation attack actually looks like.

This is what an industrial scale distillation attack actually looks like.

Autism Capital 🧩

133,088 次观看 • 5 个月前

This is some serious driving! A 420-ton distillation tower, made by Iran, is being transferred

This is some serious driving! A 420-ton distillation tower, made by Iran, is being transferred

Soureh 🟩☫🟥

426,524 次观看 • 9 个月前

The Orsknefteorgsintez refinery is burning, specifically its AVT (atmospheric-vacuum distillation) unit. This core facility processes crude oil into primary fuels.

The Orsknefteorgsintez refinery is burning, specifically its AVT (atmospheric-vacuum distillation) unit. This core facility processes crude oil into primary fuels.

NOELREPORTS 🇪🇺 🇺🇦

145,672 次观看 • 8 个月前

Asked if China's DeepSeek stole American IP, AI Czar David Sacks says it looks like a technique called distillation was used where a student model can "suck the knowledge" out of the parent model and there is evidence that DeepSeek distilled knowledge from OpenAI's models, which is leading to efforts to develop techniques to prevent future copycats

Asked if China's DeepSeek stole American IP, AI Czar David Sacks says it looks like a technique called distillation was used where a student model can "suck the knowledge" out of the parent model and there is evidence that DeepSeek distilled knowledge from OpenAI's models, which is leading to efforts to develop techniques to prevent future copycats

Tsarathustra

710,764 次观看 • 1 年前

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹

Yacine Mahdid

13,023 次观看 • 3 个月前

Future is colorful! EFF is the future....

Future is colorful! EFF is the future....

TheeGreat Kwedini 🇿🇦

17,174 次观看 • 2 年前

The future of the internet is interconnected. The future is omniweb.

The future of the internet is interconnected. The future is omniweb.

Demos Network

37,431 次观看 • 2 年前

The future is Digital ...the future is Streaming ... Karibu 2025.

The future is Digital ...the future is Streaming ... Karibu 2025.

Kin Kariisa🇺🇬

18,322 次观看 • 1 年前

Our co-founder Terence Tao is announcing SAIR Foundation's inaugural competition: the Mathematics Distillation Challenge. Co-organized by Damek, Terence Tao, and SAIR Foundation.

Our co-founder Terence Tao is announcing SAIR Foundation's inaugural competition: the Mathematics Distillation Challenge. Co-organized by Damek, Terence Tao, and SAIR Foundation.

SAIR

470,211 次观看 • 4 个月前

China's Al strategy: Ten times less spending, winning the customer race. The secret? Distillation. #Al #China

China's Al strategy: Ten times less spending, winning the customer race. The secret? Distillation. #Al #China

Johannes Maria

42,667 次观看 • 24 天前

data labeling is dead. long live distillation. from data to object detection endpoint in 90 seconds. link:

data labeling is dead. long live distillation. from data to object detection endpoint in 90 seconds. link:

SkalskiP

136,190 次观看 • 7 个月前

The future is now. The future is $EMP. What's next begins here.

The future is now. The future is $EMP. What's next begins here.

Empyreal

21,138 次观看 • 10 个月前

Distillation has paved the way for China and others to compete at the cutting edge of AI faster and cheaper than ever before Deepseek woke up the world to its disruptive potential Our latest TechCheck Take deep dive on AI's distillation problem

Distillation has paved the way for China and others to compete at the cutting edge of AI faster and cheaper than ever before Deepseek woke up the world to its disruptive potential Our latest TechCheck Take deep dive on AI's distillation problem

Deirdre Bosa

11,694 次观看 • 1 年前

Claude distillation has been a big topic this week while I am (coincidentally) writing Chapter 8 on model distillation. In that context, I shared some utilities to generate distillation data from all sorts of open-weight models via OpenRouter and Ollama:

Claude distillation has been a big topic this week while I am (coincidentally) writing Chapter 8 on model distillation. In that context, I shared some utilities to generate distillation data from all sorts of open-weight models via OpenRouter and Ollama:

Sebastian Raschka

62,838 次观看 • 5 个月前

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."

dnap

190,421 次观看 • 8 天前

AI Agents are the Future 🔮 This is the future. This is Searchpad.

AI Agents are the Future 🔮 This is the future. This is Searchpad.

Searchpad

139,012 次观看 • 1 年前

Chris Marker's LA JETÉE is storytelling pared all the way back, an inventive distillation of cinema that demands you lean in to the image and inspect every frame.

Chris Marker's LA JETÉE is storytelling pared all the way back, an inventive distillation of cinema that demands you lean in to the image and inspect every frame.

MUBI

16,918 次观看 • 1 年前

WHY $ASTER ? Decentralization is the future and ASTER IS LEADING THE FUTURE OF DEXS

WHY $ASTER ? Decentralization is the future and ASTER IS LEADING THE FUTURE OF DEXS

Julia

24,474 次观看 • 9 个月前

The future should look like the future Robovan is 🤯

The future should look like the future Robovan is 🤯

Tesla Owners Silicon Valley

63,667,248 次观看 • 8 个月前