正在加载视频...
视频加载失败
Switch Transformer by hand ✍️ ~ 13 steps walkthrough below The Switch Transformer, by Fedus, Zoph, and Shazeer in 2022, is one of the papers that made sparse Mixture of Experts practical at scale. Today, frontier models use MoE to pack enormous parameter counts while activating only a small... show more
3 条评论
Martin Ronfort3 个月前
MoE made dense parameter scaling practical. What's remarkable now is seeing active parameter counts scale down so far without losing capability.
Vítor - e/acc3 个月前
Beautiful
Henry Nguyen3 个月前
Hands-on breakdowns like this are how MoE finally clicks—sparse routing feels way less magic once you trace the dispatch/combine steps yourself.
![Self Attention by hand ✍️ ~ 9 steps walkthrough below Self-attention is what enables LLMs to understand context. How does it work? So I drew and calculated one entirely by hand. Goal: turn four 6D features into four 3D attention weighted features, filling in every cell yourself. = 1. Given = Four feature vectors, six dimensions each, one per position. = 2. Query, key, value = Let us multiply the features by WQ, WK and WV. Queries, keys and values all come out of the same four features, and that is what the word "self" is doing in self-attention. = 3. Prepare for MatMul = We copy the queries across the top and the transposed keys down the side. Lining the two up is half the work. = 4. MatMul = Let us multiply K transpose by Q. Every cell is the dot product of one key with one query, which we use as a matching score. That works because the dot product is the numerator of cosine similarity: it is how alike two vectors are, before anyone divides by their lengths. = 5. Scale = We divide by the square root of dk, the dimension of a key vector, here 3. Without it the scores grow with the dimension and a 64-wide head would swamp the softmax. To keep the page doable in pen, the drawing approximates dividing by root 3 with halving. = 6. e to the power = Let us raise e to the power of each score. This is the first half of softmax, and the drawing uses 3 in place of e, which is close enough to do in your head. = 7. Sum = We add up each column: 16, 6, 7 and 12. = 8. Normalize = Let us divide every cell by its column sum. That gives the attention weight matrix in yellow, and each of its four columns is now a probability distribution over the four positions. The decimals are nudged as they are rounded, so every column still sums to exactly 1. = 9. MatMul = We multiply the value vectors by those weights. Each output is a blend of all four values, mixed in the proportion the attention matrix just decided, and it goes to the position-wise feed forward network in the next layer: the FFN box at the bottom of the page. The outputs: Attention weights (A), by column = [.2, .6, 0, .2], [.2, .4, .2, .2], [.4, .2, 0, .4], [.1, .7, .1, .1] Attention weighted features (Z) = [8, 2, 6], [8, 4, 4], [16, 4, 2], [4, 2, 7] The takeaway: attention is a weighted average, and everything before step 9 exists to decide the weights. Compare every position with every other, turn the scores into one distribution per position, then blend. 💾 Save this post!](https://image.24vids.com/tw-2079933315990057242/ext_tw_video_thumb/2079933302048219136/pu/img/q3wr-ymUWG3Ai-ij.jpg)
![[Self-Attention] by Hand ✍️ Self-attention is what enables LLMs to understand context. How does it work? This exercise demonstrates how to calculate a 6-3 attention head by hand. Note that if we have two instances of this, we get 6-6 attention (i.e., multi-head attention, n=2). -- 𝗚𝗼𝗮𝗹 -- Transform [6D Features 🟧] to [3D Attention Weighted Features 🟦] -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- [1] Given ↳ A set of 4 feature vectors (6-D): x1,x2,x3,x4 [2] Query, Key, Value ↳ Multiply features x's with linear transformation matrices WQ, WK, and WV, to obtain query vectors (q1,q2,q3,q4), key vectors (k1,k2,k3,k4), and value vectors (v1,v2,v3,v4). ↳ "Self" refers to the fact that both queries and keys are derived from the same set of features. [3] 🟪 Prepare for MatMul ↳ Copy query vectors ↳ Copy the transpose of key vectors [4] 🟪 MatMul ↳ Multiply K^T and Q ↳ This is equivalent to taking dot product between every pair of query and key vectors. ↳ The purpose is to use dot product as an estimate of the "matching score" between every key-value pair. ↳ This estimate makes sense because dot product is the numerator of Cosine Similarity between two vectors. [5] 🟨 Scale ↳ Scale each element by the square root of dk, which is the dimension of key vectors (dk=3). ↳ The purpose is to normalize the impact of the dk on matching scores, even if we scale dk to 32, 64, or 128. ↳ To simplify hand calculation, we approximate [ □/sqrt(3) ] with [ floor(□/2) ]. [6] 🟩 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [7] 🟩 Softmax: ∑ ↳ Sum across each column [8] 🟩 Softmax: 1 / sum ↳ For each column, divide each element by the column sum ↳ The purpose is normalize each column so that the numbers sum to 1. In other words, each column is a probability distribution of attention, and we have four of them. ↳ The result is the Attention Weight Matrix (A) (yellow) [9] 🟦 MatMul ↳ Multiply the value vectors (Vs) with the Attention Weight Matrix (A) ↳ The results are the attention weighted features Zs. ↳ They are fed to the position-wise feed forward network in the next layer.](https://image.24vids.com/tw-1797249951325434315/ext_tw_video_thumb/1797249707795697665/pu/img/bRQFS-RcgUFbmOnk.jpg)
![[RLHF] by Hand ✍️ Yesterday, Jan Leike (Jan Leike) announced he is joining #Anthropic to lead their "super-alignment" mission. He is the co-inventor of Reinforcement Learning with Human Feedback (#RLHF). How does RLHF work? [1] Given ↳ Reward Model (RM) ↳ Large Language Model (LLM) ↳ Two (Prompt, Next) Pairs 🟪 TRAIN RM Goal: Learn to give higher rewards to winners [2] Preferences ↳ A human reviews the two pairs and picks a "winner" ↳ (doc is, him) Embeddings ↳ This prompt has never received human feedback directly ↳ [S] is the special start symbol [11] Transformer ↳ Attention (yellow) ↳ Feed Forward (4x2 weight and bias matrix) ↳ Output: 3 "transformed" feature vector, one per position ↳ More details in my previous post 8. Transformer [] [12] Output Probabilities ↳ Apply a linear layer to map each transformed feature vector to a probability distribution over the vocabulary. [13] Sample ↳ Apply the greedy method, which is to pick the word with the highest score ↳ For output 1 and 2, the model accurately predicts the next word ↳ For 3rd output position, the model's predicts "him" [14] Reward Model ↳ The new pair (CEO is, him) is fed to the reward model ↳ The process is same as [3]-[6] ↳ Output: Reward = 3 [15] Loss Gradient ↳ We set the loss as the negative of the reward. ↳ The loss gradient is simply a constant -1. ↳ Run backpropagation and gradient descent to update LLM's weights and biases (red border)](https://image.24vids.com/tw-1795803665845899511/ext_tw_video_thumb/1795803020149534720/pu/img/jb8SBLT_1drHtYd_.jpg)
![SORA by Hand ✍️ OpenAI’s #SORA took over the Internet when it was announced earlier this year. The technology behind Sora is the Diffusion Transformer (DiT) developed by William Peebles and Shining Xie. How does DiT work? 𝗚𝗼𝗮𝗹: Generate a video conditioned by a text prompt and a series of diffusion steps [1] Given ↳ Video ↳ Prompt: "sora is sky" ↳ Diffusion step: t = 3 [2] Video → Patches ↳ Divide all pixels in all frames into 4 spacetime patches [3] Visual Encoder: Pixels 🟨 → Latent 🟩 ↳ Multiply the patches with weights and biases, followed by ReLU ↳ The result is a latent feature vector per patch ↳ The purpose is dimension reduction from 4 (2x2x1) to 2 (2x1). ↳ In the paper, the reduction is 196,608 (256x256x3)→ 4096 (32x32x4) [4] ⬛ Add Noise ↳ Sample a noise according to the diffusion time step t. Typically, the larger the t, the smaller the noise. ↳ Add the Sampled Noise to latent features to obtain Noised Latent. ↳ The goal is to purposely add noise to a video and ask the model to guess what that noise is. ↳ This is analogous to training a language model by purposely deleting a word in a sentence and ask the model to guess what the deleted word was. [5-7] 🟪 Conditioning by Adaptive Layer Norm [5] Encode Conditions ↳ Encode "sora is sky" into a text embedding vector [0,1,-1]. ↳ Encode t = 3 to as a binary vector [1,1]. ↳ Concatenate the two vectors in to a 5D column vector. [6] Estimate Scale/Shift ↳ Multiply the combined vector with weights and biases ↳ The goal is to estimate the scale [2,-1] and shift [-1,5]. ↳ Copy the result to (X) and (+) [7] Apply Scale/Sift ↳ Scale the noised latent by [2,-1] ↳ Shifted the scaled noised latent by [-1, 5] ↳ The result is "conditioned" noise latent. [8-10] Transformer [8] Self-Attention ↳ Feed the conditioned noised latent to Query-Key function to obtain a self-attention matrix ↳ Value is omitted for simplicity [9] Attention Pooling ↳ Multiply the conditioned noised latent with the self-attention matrix ↳ The result are attention weighted features [10] Pointwise Feed Forward Network ↳ Multiply the attention weighted features with weights and biases ↳ The result is the Predicted Noise 🏋️♂️ 𝗧𝗿𝗮𝗶𝗻 [11] ↳ Calculate MSE loss gradients by taking the different between the Predicted Noise and the Sampled Noise (ground truth). ↳ Use the loss gradients to kick off backpropagation to update all learnable parameters (red borders) ↳ Note the visual encoder and decoder's parameters are frozen (blue borders) 🎨 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗲 (𝗦𝗮𝗺𝗽𝗹𝗲) [12] Denoise ↳ Subtract the predicted noise from the noised latent to obtain the noise-free latent [13] Visual Decoder: Latent 🟩 → Pixels 🟨 ↳ Multiply the patches with weights and biases, followed by ReLU [14] Patches → Video ↳ Rearrange patches into a sequence of video frames.](https://image.24vids.com/tw-1795449683285848509/ext_tw_video_thumb/1795449360802525185/pu/img/j-hppkMq667X9mjK.jpg)
![Graph Convolutional Network by hand ✍️ ~ 12 steps walkthrough below Graph Convolutional Networks (GCNs), introduced by Thomas Kipf and Max Welling in 2017, are the tool for data shaped like a graph: social networks, recommendations, biological networks, drug discovery, molecular chemistry. I drew and calculated a simple GCN entirely by hand. Goal: run a two-layer GCN, then a small classifier, on a five-node graph, filling in every cell yourself. 1. Given A graph of five nodes, A to E, with edges between some of them. 2. Adjacency matrix (neighbors) Put a 1 wherever two nodes share an edge, in both directions. 3. Adjacency matrix (self) Add 1s down the diagonal, one self-loop per node. That is just adding the identity matrix. 4. Messages Multiply each node's embedding by the weights and biases, then ReLU. Negatives become 0. 5. Pooling Multiply the messages by the adjacency matrix. Each node gathers the messages of its neighbours and itself. 6. Visualize Node A pools [3,0,1] + [1,0,0] = [4,0,1]. 7. Second GCN layer Messages again: weights, biases, ReLU. 8. Pooling again Pool over each node and its neighbours, once more. 9. Visualize Node C pools [1,2,4] + [1,3,5] + [0,0,1] = [2,5,10]. 10. Fully connected layer Weights, biases, ReLU. This time there are no neighbours to pool, just the node itself. 11. Linear layer One more: weights and biases. 12. Sigmoid Squash each score to a probability (≥ 3 → 1, 0 → 0.5, ≤ -3 → 0). That is the classification for each node. You have just classified every node in the graph by hand. ✍️ The outputs: A: 0 (very unlikely) B: 1 (very likely) C: 1 (very likely) D: 1 (very likely) E: 0.5 (neutral) The takeaway: a GCN layer is two parts. The top part pools each node with its neighbours through the adjacency matrix. The bottom part is an MLP that transforms each node on its own. A transformer layer has the same two parts, with an attention matrix where the adjacency matrix was. Both matrices do one job, mixing across positions: attention over tokens, adjacency over nodes. In my class I call the GCN the transformer's little cousin: a bit more stubborn, because its attention is fixed by the graph rather than computed from Q, K, and V. Draw the two side by side and the resemblance is hard to miss. 💾 Save this post! #AIbyHand #GraphNeuralNetworks #DeepLearning](https://image.24vids.com/tw-2077396566864773414/ext_tw_video_thumb/2077396555410051073/pu/img/JX9U_Rk-wmgd1S-x.jpg)
![SVM by hand ✍️ ~ 19 steps walkthrough below (Linear vs RBF) Support Vector Machines reigned supreme in machine learning before the deep learning revolution. An SVM predicts with dot products, the same matrix multiplication every model uses. What it does not do is train by backpropagation: it is fitted by convex optimization, so there is no matrix-multiplication backward pass for a GPU to accelerate. I drew and calculated two SVMs by hand: a linear one (top) and an RBF one (bottom), classifying the same two test vectors. Goal: turn six training vectors and their learned coefficients into a prediction, and see what changing the kernel actually changes. = 1. Given = Six training vectors, their labels, and the coefficients and bias already learned. A coefficient of zero means that vector is not a support vector: too far from the boundary to matter. = 2. Linear kernel, test vector 1 = Let us take the dot product of the test vector with every training vector. The dot product stands in for cosine similarity, and the column of results is the first column of the kernel matrix K. = 3. Linear kernel, test vector 2 = We do the same for the second, and K is complete. = 4. Signed weights = Let us multiply each coefficient by its label. The second training vector drops out here, because its coefficient is 0. = 5. Weighted combination = We multiply the signed weights through K and add the bias b. The result is a signed distance to the decision boundary: 17 and 5. = 6. Classify = Let us take the sign. Both are positive. = 7 to 11. RBF kernel, test vector 1 = Now the same picture with a different kernel, in five moves: square the differences, sum them, take the square root for the L2 distance, multiply by minus gamma, and raise e to that power. The negation is what turns a distance into a similarity, and gamma controls how far a single training vector's influence reaches. = 12 to 16. RBF kernel, test vector 2 = We repeat all five. The numbers change, the moves do not. = 17 to 19. Decision boundary, again = Signed weights, weighted combination, sign. Identical arithmetic to steps 4 through 6, on a K that was built a completely different way. The outputs: Linear K, first column = [13, 25, 12, 15, 19, 27] Linear decision values = 17 and 5, both positive RBF decision values = -2 and 1, so negative and positive The takeaway: the kernel is the only thing that changed, and it changed the answer. The linear SVM calls both test vectors positive; the RBF one splits them. Everything after the kernel matrix, the signed weights and the weighted combination and the sign, is the same page of arithmetic twice. 💾 Save this post!](https://image.24vids.com/tw-2080658072842703031/ext_tw_video_thumb/2080658053083394048/pu/img/Lrl4Xyr6GSkbbbrN.jpg)
![Dropout by hand ✍️ ~ 10 steps walkthrough below Dropout is the simplest trick in deep learning that actually works: during training you randomly switch neurons off, so the network cannot lean on any one of them. It is two lines of code and almost nobody has worked through what those lines do to the numbers. So I drew and calculated one entirely by hand. Goal: train one pass through a small network with two dropout layers, then run inference with dropout switched off. The network: Linear(2,4), ReLU, Dropout(0.5), Linear(4,3), ReLU, Dropout(0.33), Linear(3,2). = 1. Given = A training set of two examples, X1 and X2, and the weight matrices for all three linear layers. = 2. Draw the first random numbers = Let us draw 4 random numbers, one per neuron in the first hidden layer. Above 0.5 we keep (◯), below we drop (╳). Here that gives [◯, ╳, ◯, ╳]. = 3. Build the first dropout matrix = We turn that pattern into a diagonal matrix. The scaling factor is 1/(1-p) = 2, so a kept neuron gets 2 and a dropped one gets 0. Multiplying by it does both jobs at once: it deletes the 2nd and 4th neurons and doubles the two that survive. = 4. Draw the second random numbers = Let us do it again for the 3 neurons in the next layer, this time against p = 0.33. The result is [◯, ◯, ╳]. = 5. Build the second dropout matrix = We set the diagonal to 1.5 where kept and 0 where dropped. Only the 3rd neuron goes. = 6. Feed forward = Let us run the whole thing top to bottom: one matrix multiplication per layer, ReLU setting the negatives to zero, and the two dropout matrices doing their work in between. The outputs Y come out at the bottom. = 7. MSE loss gradients = We compare Y against the targets Y', subtract, and multiply each element by 2. That is the whole gradient of the mean squared error. = 8. Update the weights = Let us push those gradients back through the network and update the weights (marked in light red). = 9. Deactivate dropout = Training is over, so we set both dropout matrices to the identity. Every neuron is back, and nothing is scaled. = 10. Feed forward again = One more pass, this time on unseen data, to make the prediction. You have just trained and run a network with dropout by hand. ✍️ The outputs: Training outputs Y = [-6, 9; 13, 4] Loss gradients = [-4, 4; 6, -2] Inference outputs = [13, 13; 4, 3] 💾 Save this post! #AIbyHand #Dropout #DeepLearning #NeuralNetworks](https://image.24vids.com/tw-2078846104460783996/ext_tw_video_thumb/2078846092997763074/pu/img/pljAsdaJjvGEEjPE.jpg)
![Discrete Fourier Transform by hand ✍️ ~ 12 steps walkthrough below Here is a little-known secret about the DFT and the inverse DFT: it is just matrix multiplication in both directions, one the transpose of the other, exactly like the forward pass and backpropagation I drew in other examples. Goal: recover which cosine waves a signal is made of, using nothing but multiplication and addition. = 1. Given = Three signals written as sums of cosines, and a fourth, X, that we do not know yet. = 2. Frequency matrix F = Let us write the coefficients as a matrix. Each signal is a row, each frequency a column, so A = cos(w) + 2cos(2w) becomes [1, 2, 0, 0]. = 3. Sample the waves = We read the four cosine waves at ten discrete time points. That word "discrete" is the whole difference between this and the continuous transform. = 4. Cosine matrix W = Let us write those samples as a matrix: each frequency a row, each time point a column. = 5. Frequency to time = We multiply F by W. That combines the four cosine waves in the proportions F specifies, and the result T is the three signals as they would look in time. = 6. Transpose = Let us stand each signal up as a column. = 7. Time to frequency = We multiply W by that transpose. Every cell is the dot product of one signal with one cosine wave, which measures how much of that wave the signal contains. Zero means none of it. = 8. Scale = Let us multiply by 2/n, with n = 10. The projections come out five times too large, and this is the correction. = 9. Transpose back = We turn it back around, and it is F again, exactly. That is the check: the transform recovered the coefficients we started from. = 10. Now solve for X = Let us run the same multiplication on the one signal whose recipe we never knew. = 11. Scale = We divide by 5 again. = 12. Transpose back = And X reads [0, 0, 3, 2], which says X = 3cos(3w) + 2cos(4w). Note: I originally drew this to show that the DFT is a special case of a convolution layer, its filters fixed to sine and cosine waves rather than learned. No wonder, then, that a convolution layer free to learn its own filters can be trained to process signals. 💾 Save this post!](https://image.24vids.com/tw-2096604730453577857/ext_tw_video_thumb/2096604718290120705/pu/img/Uxa9xebSM4EBcEzP.jpg)
![Discrete Fourier Transform by hand ✍️ ~ 12 steps walkthrough below Here is a little-known secret about the DFT and the inverse DFT: it is just matrix multiplication in both directions, one the transpose of the other, exactly like the forward pass and backpropagation I drew in other examples. Goal: recover which cosine waves a signal is made of, using nothing but multiplication and addition. = 1. Given = Three signals written as sums of cosines, and a fourth, X, that we do not know yet. = 2. Frequency matrix F = Let us write the coefficients as a matrix. Each signal is a row, each frequency a column, so A = cos(w) + 2cos(2w) becomes [1, 2, 0, 0]. = 3. Sample the waves = We read the four cosine waves at ten discrete time points. That word "discrete" is the whole difference between this and the continuous transform. = 4. Cosine matrix W = Let us write those samples as a matrix: each frequency a row, each time point a column. = 5. Frequency to time = We multiply F by W. That combines the four cosine waves in the proportions F specifies, and the result T is the three signals as they would look in time. = 6. Transpose = Let us stand each signal up as a column. = 7. Time to frequency = We multiply W by that transpose. Every cell is the dot product of one signal with one cosine wave, which measures how much of that wave the signal contains. Zero means none of it. = 8. Scale = Let us multiply by 2/n, with n = 10. The projections come out five times too large, and this is the correction. = 9. Transpose back = We turn it back around, and it is F again, exactly. That is the check: the transform recovered the coefficients we started from. = 10. Now solve for X = Let us run the same multiplication on the one signal whose recipe we never knew. = 11. Scale = We divide by 5 again. = 12. Transpose back = And X reads [0, 0, 3, 2], which says X = 3cos(3w) + 2cos(4w). Note: I originally drew this to show that the DFT is a special case of a convolution layer, its filters fixed to sine and cosine waves rather than learned. No wonder, then, that a convolution layer free to learn its own filters can be trained to process signals. 💾 Save this post!](https://image.24vids.com/tw-2083194764485153128/ext_tw_video_thumb/2083194751193403393/pu/img/xPTCxxe0pHHx3B7I.jpg)
