Loading video...

Video Failed to Load

Go Home

137,110 views • 8 months ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Revisiting this interesting story during Nobel Prize week: In the heart of World War II, as the Nazis took control of Copenhagen, a peculiar situation took place at the Institute of Theoretical Physics, led by physicist Niels Bohr. Two Nobel laureates Max von Laue and James Franck, fearing the confiscation of their gold Nobel Prize medals by the Nazis, had sent their medals to Bohr for safekeeping. On the day the Nazis arrived in Copenhagen, Hungarian chemist Georgy de Hevesy, who was working in Bohr's lab, devised a plan to prevent the discovery of the medals. Initially considering burying the medals, they quickly dismissed the idea, fearing the thorough searches the Nazis would conduct. Instead, de Hevesy proposed a chemical solution — literally. Utilizing a mixture known as "aqua regia" (a blend of hydrochloric and nitric acids), he set about dissolving the gold medals. This concoction is one of the few substances capable of dissolving gold, a notably unreactive element. As the Nazis marched outside, de Hevesy dissolved the precious medals, reducing them to a colorless solution that eventually turned bright orange. The liquid containing the dissolved gold was then placed on a high shelf in the laboratory, where it remained unnoticed throughout the Nazi occupation​. Post World War II, upon returning to the laboratory after V-E Day, de Hevesy found the beaker undisturbed on the shelf. The gold was recovered from the solution and returned to the Nobel Prize committee, who then reminted the medals and presented them back to Laue and Franck in a ceremony in 1952.

Fermat's Library

127,557 views • 1 year ago

Full Fine-tuning vs. Freezing Layers. Interact 👉 and == Full Fine-tuning == A real network has many — three layers in this example, billions of parameters in a production model. What does fine-tuning look like when you update all of them? That’s full fine-tuning: continue training every weight in the pretrained network on your new task. Every layer’s W gets its own ΔW. Nothing is frozen — every parameter is in play. Think of an MLP as a chain of prerequisites leading to an advanced course. Layer 1 might be Linear Algebra, layer 2 Probability, layer 3 Advanced Machine Learning — each one building on what came before. Fine-tuning is what happens during graduate study: the foundations are already there from undergrad, so you’re not re-learning. Full fine-tuning is reviewing every prerequisite to see what new topics have appeared and what discoveries the field has made since the last time you sat through them. Effective — but exhausting. This diagram shows the same three-layer MLP twice, side by side. On the left, the pretrained network runs on input X: three weight matrices W₁, W₂, W₃, each followed by a ReLU activation. Full fine-tuning gives the model the most freedom to specialize. Every parameter can move — and every parameter that can move must be stored. But not every prerequisite needs revisiting. The further you go back in the chain, the less the material has changed since pretraining — the linear-algebra basics under your computer-vision course are largely the same as they ever were. The next page does exactly that: freeze the prerequisites that haven’t moved, and only refresh the advanced one closest to your specialization. == Freezing Layers == Full fine-tuning reviewed every prerequisite — Linear Algebra, Probability, Advanced ML — to refresh each subject with the latest topics. Effective, but exhausting. Then you realize something. The prerequisites haven’t actually changed that much. Linear Algebra is still Linear Algebra; the matrix decompositions you learned still hold. Probability is still Probability; the distributions and Bayes’ rule haven’t moved. Almost all the new material — the new ideas, the recent discoveries — lives in the advanced layer at the top. That’s freezing layers: keep the prerequisite layers fixed at their pretrained state, and only update the advanced one. In the diagram below, W1​ and W2​ — the foundational prerequisites — stay frozen. Only W3​ — the layer closest to your task-specific output — gets a ΔW.

Tom Yeh

27,587 views • 3 months ago