
Imbue
@imbue_ai • 9,532 subscribers
We build honest software
Videos

Deep learning works extraordinarily well. And we still largely don't know why. A new paper from Jamie Simon, Daniel Kunin, and 12 co-authors argues that a scientific theory of deep learning is emerging, and coins a name for the emerging field: learning mechanics. We sat down with Jamie and Dan on Generally Intelligent to talk about what a physics of deep learning would actually look like, why now, and what's left to figure out. 3:05 Learning mechanics as the physics to mechanistic interpretability's biology 4:13 Why deep learning needs a theory 7:07 Why deep learning is uniquely hard to engineer 12:11 How a week in the woods became a paper 25:59 The barrier to theory isn't opacity, but complexity 36:26 Deep learning's first gas law 47:22 Why more particles makes the problem easier 56:22 The discretization hypothesis 1:01:50 The strongest signal that a compact theory exists 1:05:07 The Platonic Representation Hypothesis 1:15:41 Why learning mechanics and mech interp need each other 1:25:29 Theory as safety infrastructure
Imbue17,877 次观看 • 1 个月前

🎙️ New Generally Intelligent podcast episode! We interviewed Tri Dao (author of FlashAttention & Chief Scientist Together AI) and discussed: - Taking a contrarian bet on recurrent connections over attention - Using data augmentation to encode knowledge into models - Designing algorithms that take advantage of hardware ...and more! Podcast links and highlights:
Imbue19,151 次观看 • 2 年前
没有更多内容可加载


