Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

[Backpropagation] by Hand✍️ [1] Forward Pass ↳ Given a multi layer perceptron (3 levels), an input vector X, predictions Y^{Pred} = [0.5, 0.5, 0], and ground truth label Y^{Target} = [0, 1, 0]. [2] Backpropagation ↳ Insert cells to hold our calculations. [3] Layer 3 - Softmax (blue) ↳... Calculate ∂L / ∂z3 directly using the simple equation: Y^{Pred} - Y^{Target} = [0.5, -0.5, 0]. ↳ This simple equation is the benefit of using Softmax and Cross Entropy Loss together. [4] Layer 3 - Weights (orange) & Biases (black) ↳ Calculate ∂L / ∂W3 and ∂L / ∂b3 by multiplying ∂L / ∂z3 and [ a2 | 1 ]. [5] Layer 2 - Activations (green) ↳ Calculate ∂L / ∂a2 by multiplying ∂L / ∂z3 and W3. [6] Layer 2 - ReLU (blue) ↳ Calculate ∂L / ∂z2 by multiplying ∂L / ∂a2 with 1 for positive values and 0 otherwise. [7] Layer 2 - Weights (orange) & Biases (black) ↳ Calculate ∂L / ∂W2 and ∂L / ∂b2 by multiplying ∂L / ∂z2 and [ a1 | 1 ]. [8] Layer 1 - Activations (green) ↳ Calculate ∂L / ∂a1 by multiplying ∂L / ∂z2 and W2. [9] Layer 1 - ReLU (blue) ↳ Calculate ∂L / ∂z1 by multiplying ∂L / ∂a1 with 1 for positive values and 0 otherwise. [10] Layer 1 - Weights (orange) & Biases (black) ↳ Calculate ∂L / ∂W1 and ∂L / ∂b1 by multiplying ∂L / ∂z1 and [ x | 1 ]. [11] Gradient Descent ↳ Update weights and biases (typically a learning rate is applied here). 💡 Matrix Multiplication is All You Need: Just like in the forward pass, backpropagation is all about matrix multiplications. You can definitely do everything by hand as I demonstrated in this exercise, albeit slow and imperfect. This is why GPU's ability to multiply matrices efficiently plays such an important role in the deep learning evolution. This is why NVIDIA is now close to $1 trillion in valuation. 💡Exploding Gradients: We can already see the gradients are getting larger as we back-propagate up, even in this simple 3-layer network. This motivates using methods like skip connections to handle exploding (or diminishing) gradients as in the ResNet. I did the calculations entirely by hand. Please let me know if you spot any error or have any questions!show more

Tom Yeh

65,578 subscribers

64,783 Aufrufe • vor 2 Jahren •via X (Twitter)

Gesundheit & Wellness Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

[Deep RNN] by Hand ✍️ A Deep Recurrent Neural Network (RNN) extends a basic single-layer RNN into multiple layers of hidden states, effectively incorporating deep learning into the RNN architecture. How does a Deep RNN work? [1] Given ↳ A sequence of four inputs X1, X2, X3, X4 ⬛️ ↳ Recurrent weights and biases for hidden layers a 🟩, b 🟧, c 🟪, and the output layer y 🟦. [2] Initialize Hidden States ↳ Set a0, b0, c0 to zeros — Process X1 (t = 1)— [3] First Hidden Layer (a) 🟩: a0 → a1 ↳ The transformation matrix is horizontal concatenation of input weights, hidden state weights and biases, visualized as [⬛️ | 🟩 | ⬜️] . ↳ The state matrix is vertical concatenation of input X1, previous hidden state a0, and an extra 1, visualized as [⬛️ ; 🟩 ; 1]. ↳ Multiply the two matrices to obtain new hidden state a1 = [0 ; 1]. [4] Second Hidden Layer (b) 🟪: b0 → b1 ↳ First layer a1 🟩 becomes the input. ↳ The transformation matrix is visualized as [🟩 | 🟪 | ⬜️]. ↳ The state matrix is the combination of a1, b0, and 1, visualized as [🟩; 🟪 ; 1]. ↳ Multiply the two matrices to obtain new hidden state b1 = [1; -1]. [5] Third Hidden Layer (c) 🟧: c0 → c1 ↳ Second layer b 🟪 becomes the input. ↳ The transformation matrix is visualized as [🟪 | 🟧 | ⬜️]. ↳ The state matrix is the combination of a1, b0, and 1, visualized as [🟪; 🟧; 1]. ↳ Multiply the two matrices to obtain new hidden state b1 = [1; -1]. [6] Output Layer (Y) 🟦 ↳ The transformation matrix is visualized as [🟧 | ⬜️]. ↳ The state matrix is the combination of c0 and , visualized as [🟧; 1]. ↳ Multiply the two matrices to obtain output Y1 = [3; 0; 3]. — Process X2 (t = 2)— [7] Previous Hidden States ↳ Copy the values of a1, b1, c1. [8] Hidden 🟩🟪🟧 + Output 🟦 ↳ Repeat [3]-[6] to obtain output Y2 = [5; 0; 4] — Process X3 (t = 3)— [9] Previous Hidden States ↳ Copy the values of a2, b2, c2. [10] Hidden 🟩🟪🟧 + Output 🟦 ↳ Repeat [3]-[6] to obtain output Y3 = [13; -1; 9] — Process X4 (t = 4)— [11] Previous Hidden States ↳ Copy the values of a3, b3, c3. [12] Hidden 🟩🟪🟧 + Output 🟦 ↳ Repeat [3]-[6] to obtain output Y4 = [15; 7; 2]

[Deep RNN] by Hand ✍️ A Deep Recurrent Neural Network (RNN) extends a basic single-layer RNN into multiple layers of hidden states, effectively incorporating deep learning into the RNN architecture. How does a Deep RNN work? [1] Given ↳ A sequence of four inputs X1, X2, X3, X4 ⬛️ ↳ Recurrent weights and biases for hidden layers a 🟩, b 🟧, c 🟪, and the output layer y 🟦. [2] Initialize Hidden States ↳ Set a0, b0, c0 to zeros — Process X1 (t = 1)— [3] First Hidden Layer (a) 🟩: a0 → a1 ↳ The transformation matrix is horizontal concatenation of input weights, hidden state weights and biases, visualized as [⬛️ | 🟩 | ⬜️] . ↳ The state matrix is vertical concatenation of input X1, previous hidden state a0, and an extra 1, visualized as [⬛️ ; 🟩 ; 1]. ↳ Multiply the two matrices to obtain new hidden state a1 = [0 ; 1]. [4] Second Hidden Layer (b) 🟪: b0 → b1 ↳ First layer a1 🟩 becomes the input. ↳ The transformation matrix is visualized as [🟩 | 🟪 | ⬜️]. ↳ The state matrix is the combination of a1, b0, and 1, visualized as [🟩; 🟪 ; 1]. ↳ Multiply the two matrices to obtain new hidden state b1 = [1; -1]. [5] Third Hidden Layer (c) 🟧: c0 → c1 ↳ Second layer b 🟪 becomes the input. ↳ The transformation matrix is visualized as [🟪 | 🟧 | ⬜️]. ↳ The state matrix is the combination of a1, b0, and 1, visualized as [🟪; 🟧; 1]. ↳ Multiply the two matrices to obtain new hidden state b1 = [1; -1]. [6] Output Layer (Y) 🟦 ↳ The transformation matrix is visualized as [🟧 | ⬜️]. ↳ The state matrix is the combination of c0 and , visualized as [🟧; 1]. ↳ Multiply the two matrices to obtain output Y1 = [3; 0; 3]. — Process X2 (t = 2)— [7] Previous Hidden States ↳ Copy the values of a1, b1, c1. [8] Hidden 🟩🟪🟧 + Output 🟦 ↳ Repeat [3]-[6] to obtain output Y2 = [5; 0; 4] — Process X3 (t = 3)— [9] Previous Hidden States ↳ Copy the values of a2, b2, c2. [10] Hidden 🟩🟪🟧 + Output 🟦 ↳ Repeat [3]-[6] to obtain output Y3 = [13; -1; 9] — Process X4 (t = 4)— [11] Previous Hidden States ↳ Copy the values of a3, b3, c3. [12] Hidden 🟩🟪🟧 + Output 🟦 ↳ Repeat [3]-[6] to obtain output Y4 = [15; 7; 2]

Tom Yeh

26,562 Aufrufe • vor 2 Jahren

[Dropout] by hand ✍️ Dropout is a simple yet effective way of reducing overfitting and improving generalization. This by-hand exercise lets students practice calculating dropout, thereby gaining insight into its inner workings. As an additional bonus, students get to practice calculating the gradients of the Mean Square Error (MSE) loss. After the practice, students are often surprised by how simple it is. -- 𝗡𝗲𝘁𝘄𝗼𝗿𝗸 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 -- 1. Linear(2,4) 2. ReLU 3. Dropout(0.5) 4. Linear(4,3) 5. ReLU 6. Dropout(0.33) 7. Linear(3,2) -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- 🏋️ Training [1] Given ↳ A training set of 2 examples X1, X2 [2] 🟧 Random: p > 0.5 ↳ Draw 4 random numbers ↳ For each random number, if it is above 0.5, we keep and denote it as ◯. Otherwise we drop and denote it as ╳. ↳ The result is [◯, ╳, ◯, ╳] [3] 🟧 Dropout: Matrix ↳ Calculate the scaling factor: 1 / (1-p) = 2 ↳ Set the diagonal based on [◯, ╳, ◯, ╳], where ◯ = 2 and ╳ = 0 ↳ The purpose is to drop the 2nd and the 4th nodes, and scale the remaining two nodes by 2. [4] 🟦 Random: p > 0.33 ↳ Draw 3 random numbers ↳ For each random number, if it is above 0.33, we keep and denote it as ◯. Otherwise we drop and denote it as ╳. ↳ The result is [◯, ◯, ╳] [5] 🟦 Dropout: Matrix ↳ Calculate the scaling factor: 1 / (1-p) = 1.5 ↳ Set the diagonal based on [◯, ◯, ╳], where ◯ = 1.5 and ╳ = 0 ↳ The purpose is to drop the 3rd node, and scale the remaining two nodes by 1.5. [6] Feed Forward ↳ Now we have all the matrices ready across the layers, perform the feed forward pass by calculating a series of matrix multiplications from the top to the bottom ↳ ReLU activation function is applied along the way to set negative feature values to zeroes, denoted by ╳. ↳ The outputs are Y. [7] 🟥 Loss Gradients of Mean Square Error (MSE) ↳ The formula is 2 * (Y - Y') ↳ First we calculate Outputs (Y) - Targets (Y') ↳ Second we multiply each element by 2 [8] Update Weights ↳ Use loss gradients to start back propagation ↳ Update some weights (light red) ↳ The values of the new weights are for demonstration purpose only, not based on real calculation. 🔍 Inference [9] Deactivate Dropout ↳ We set both dropout matrices to identity matrices ↳ The effect is to keep all the features as is. [10] Feed Forward ↳ Take the forward pass to make predictions about unseen data.

[Dropout] by hand ✍️ Dropout is a simple yet effective way of reducing overfitting and improving generalization. This by-hand exercise lets students practice calculating dropout, thereby gaining insight into its inner workings. As an additional bonus, students get to practice calculating the gradients of the Mean Square Error (MSE) loss. After the practice, students are often surprised by how simple it is. -- 𝗡𝗲𝘁𝘄𝗼𝗿𝗸 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 -- 1. Linear(2,4) 2. ReLU 3. Dropout(0.5) 4. Linear(4,3) 5. ReLU 6. Dropout(0.33) 7. Linear(3,2) -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- 🏋️ Training [1] Given ↳ A training set of 2 examples X1, X2 [2] 🟧 Random: p > 0.5 ↳ Draw 4 random numbers ↳ For each random number, if it is above 0.5, we keep and denote it as ◯. Otherwise we drop and denote it as ╳. ↳ The result is [◯, ╳, ◯, ╳] [3] 🟧 Dropout: Matrix ↳ Calculate the scaling factor: 1 / (1-p) = 2 ↳ Set the diagonal based on [◯, ╳, ◯, ╳], where ◯ = 2 and ╳ = 0 ↳ The purpose is to drop the 2nd and the 4th nodes, and scale the remaining two nodes by 2. [4] 🟦 Random: p > 0.33 ↳ Draw 3 random numbers ↳ For each random number, if it is above 0.33, we keep and denote it as ◯. Otherwise we drop and denote it as ╳. ↳ The result is [◯, ◯, ╳] [5] 🟦 Dropout: Matrix ↳ Calculate the scaling factor: 1 / (1-p) = 1.5 ↳ Set the diagonal based on [◯, ◯, ╳], where ◯ = 1.5 and ╳ = 0 ↳ The purpose is to drop the 3rd node, and scale the remaining two nodes by 1.5. [6] Feed Forward ↳ Now we have all the matrices ready across the layers, perform the feed forward pass by calculating a series of matrix multiplications from the top to the bottom ↳ ReLU activation function is applied along the way to set negative feature values to zeroes, denoted by ╳. ↳ The outputs are Y. [7] 🟥 Loss Gradients of Mean Square Error (MSE) ↳ The formula is 2 * (Y - Y') ↳ First we calculate Outputs (Y) - Targets (Y') ↳ Second we multiply each element by 2 [8] Update Weights ↳ Use loss gradients to start back propagation ↳ Update some weights (light red) ↳ The values of the new weights are for demonstration purpose only, not based on real calculation. 🔍 Inference [9] Deactivate Dropout ↳ We set both dropout matrices to identity matrices ↳ The effect is to keep all the features as is. [10] Feed Forward ↳ Take the forward pass to make predictions about unseen data.

Tom Yeh

36,034 Aufrufe • vor 2 Jahren

Batch Normalization by hand ✍️ ~ 7 steps walkthrough below Batch normalization is common practice for improving training and achieving faster convergence. It sounds simple. But it is often misunderstood. 🤔 Does batch normalization involve trainable parameters, tunable hyper-parameters, or both? 🤔 Is batch normalization applied to inputs, features, weights, biases, or outputs? 🤔 How is batch normalization different from layer normalization? So I drew and calculated one entirely by hand. Goal: normalize a mini-batch of 4 examples to mean 0 and variance 1, then let the network scale it back. = 1. Given = A mini-batch of 4 training examples, each with 3 features. = 2. Linear layer = Let us multiply by the weights and add the biases. Batch norm sits after this, which answers the second question: what gets normalized is features, not inputs, weights or biases. = 3. ReLU = We apply the activation, and -2 becomes 0. Negative values are suppressed before any statistic is taken. = 4. Batch statistics = Let us compute the sum, mean, variance and standard deviation, one row at a time. A row is a feature and the four columns are the four examples, so every number here measures one feature against the rest of the batch. That is the "batch" in batch normalization, and it is exactly what layer normalization does not do. The statistics are rounded to whole numbers, which is what keeps the rest of the page doable in pen. = 5. Shift to mean 0 = We subtract the mean, in green. The four values in each feature now average to zero. = 6. Scale to variance 1 = Let us divide by the standard deviation, in orange. Each feature now has variance one, whatever scale it arrived at. = 7. Scale and shift = We multiply by a linear transformation and pass the result on. The diagonal and the last column are trainable, so having just forced every feature to mean 0 and variance 1, we hand the network the means to undo it. The outputs: Mean of each feature = [2, 1, 2] Std dev of each feature = [1, 1, 2] To the next layer = [2, -2, 2, 0], [-3, 3, 6, -3], [2, 0, 1, 2] The answers: 🤔 Both. The scale and shift are trainable, the statistics are not. Epsilon and the momentum on the running statistics are the hyper-parameters, and one mini-batch by hand needs neither. 🤔 Features, after the linear layer, not inputs, weights or biases. 🤔 Batch norm measures across the batch, one feature at a time. Layer norm measures across the features, one example at a time. 💾 Save this post!

Batch Normalization by hand ✍️ ~ 7 steps walkthrough below Batch normalization is common practice for improving training and achieving faster convergence. It sounds simple. But it is often misunderstood. 🤔 Does batch normalization involve trainable parameters, tunable hyper-parameters, or both? 🤔 Is batch normalization applied to inputs, features, weights, biases, or outputs? 🤔 How is batch normalization different from layer normalization? So I drew and calculated one entirely by hand. Goal: normalize a mini-batch of 4 examples to mean 0 and variance 1, then let the network scale it back. = 1. Given = A mini-batch of 4 training examples, each with 3 features. = 2. Linear layer = Let us multiply by the weights and add the biases. Batch norm sits after this, which answers the second question: what gets normalized is features, not inputs, weights or biases. = 3. ReLU = We apply the activation, and -2 becomes 0. Negative values are suppressed before any statistic is taken. = 4. Batch statistics = Let us compute the sum, mean, variance and standard deviation, one row at a time. A row is a feature and the four columns are the four examples, so every number here measures one feature against the rest of the batch. That is the "batch" in batch normalization, and it is exactly what layer normalization does not do. The statistics are rounded to whole numbers, which is what keeps the rest of the page doable in pen. = 5. Shift to mean 0 = We subtract the mean, in green. The four values in each feature now average to zero. = 6. Scale to variance 1 = Let us divide by the standard deviation, in orange. Each feature now has variance one, whatever scale it arrived at. = 7. Scale and shift = We multiply by a linear transformation and pass the result on. The diagonal and the last column are trainable, so having just forced every feature to mean 0 and variance 1, we hand the network the means to undo it. The outputs: Mean of each feature = [2, 1, 2] Std dev of each feature = [1, 1, 2] To the next layer = [2, -2, 2, 0], [-3, 3, 6, -3], [2, 0, 1, 2] The answers: 🤔 Both. The scale and shift are trainable, the statistics are not. Epsilon and the momentum on the running statistics are the hyper-parameters, and one mini-batch by hand needs neither. 🤔 Features, after the linear layer, not inputs, weights or biases. 🤔 Batch norm measures across the batch, one feature at a time. Layer norm measures across the features, one example at a time. 💾 Save this post!

Tom Yeh

20,848 Aufrufe • vor 3 Monaten

MLP in PyTorch by hand ✍️ ~ 7 steps walkthrough below Goal: fill in every blank in the PyTorch code to build a multi-layer perceptron. 1. Given Let us start with a code template on the left and the network it is supposed to build on the right. Every blank in the code can be worked out from the picture. 2. Linear layer We count: 3 features in, 4 features out. So the weight matrix is 4 by 3. There is an extra column for the biases, which means bias = T. 3. ReLU Let us apply the activation. ReLU crosses out the negatives, so -1 becomes 0. 4. Linear layer The input size is 4, because that is what the previous layer put out. The output size is 2. A 2 by 4 weight matrix, and this time no extra column, so bias = F. 5. ReLU We cross out the negatives again. 6. Linear layer Two features in, five out. A 5 by 2 weight matrix, with a bias column, so bias = T. 7. Sigmoid Let us finish. Sigmoid squashes the raw scores (3, 0, -2, 5, -5) into probabilities between 0 and 1. You have just implemented a three-layer deep neural network by hand. ✍️ == Story == Three years ago I gave this exercise to my students, to connect the code to the math. They found it odd. Every other AI course they were taking lived inside a Jupyter notebook, and here I was handing out paper. Three years later, my colleagues are the ones rushing to move their materials to paper. The exercise has not changed. Paper still asks the one thing a notebook lets you skip: do you actually understand what the code is doing? If you can tell me why the weight matrix is 4 by 3, and why bias is F on the second layer, you understand nn.Linear better than someone who has been copy-pasting it for a year. 💾 Save this post! #AIbyHand #PyTorch #DeepLearning

MLP in PyTorch by hand ✍️ ~ 7 steps walkthrough below Goal: fill in every blank in the PyTorch code to build a multi-layer perceptron. 1. Given Let us start with a code template on the left and the network it is supposed to build on the right. Every blank in the code can be worked out from the picture. 2. Linear layer We count: 3 features in, 4 features out. So the weight matrix is 4 by 3. There is an extra column for the biases, which means bias = T. 3. ReLU Let us apply the activation. ReLU crosses out the negatives, so -1 becomes 0. 4. Linear layer The input size is 4, because that is what the previous layer put out. The output size is 2. A 2 by 4 weight matrix, and this time no extra column, so bias = F. 5. ReLU We cross out the negatives again. 6. Linear layer Two features in, five out. A 5 by 2 weight matrix, with a bias column, so bias = T. 7. Sigmoid Let us finish. Sigmoid squashes the raw scores (3, 0, -2, 5, -5) into probabilities between 0 and 1. You have just implemented a three-layer deep neural network by hand. ✍️ == Story == Three years ago I gave this exercise to my students, to connect the code to the math. They found it odd. Every other AI course they were taking lived inside a Jupyter notebook, and here I was handing out paper. Three years later, my colleagues are the ones rushing to move their materials to paper. The exercise has not changed. Paper still asks the one thing a notebook lets you skip: do you actually understand what the code is doing? If you can tell me why the weight matrix is 4 by 3, and why bias is F on the second layer, you understand nn.Linear better than someone who has been copy-pasting it for a year. 💾 Save this post! #AIbyHand #PyTorch #DeepLearning

Tom Yeh

13,318 Aufrufe • vor 3 Monaten

[Graph Convolutional Network] by hand ✍️ Graph Convolutional Networks (GCNs), introduced by Thomas Kipf and Max Welling in 2017, have emerged as a powerful tool in the analysis and interpretation of data structured as graphs. This exercise demonstrates how GCN works in a simple application: binary classification. -- Goal -- Predict if a node in a graph is X. -- Architecture -- 🟪 Graph Convolutional Network (GCN) 1. GCN1(4,3) 2. GCN2(3,3) 🟦 Fully Connected Network (FCN) 1. Linear1(3,5) 2. ReLU 3. Linear2(5,1) 4. Sigmoid Simplications: • Adjacent matrices are not normalized. • ReLU is applied to messages directly. -- Walkthrough -- [1] Given ↳ A graph with five nodes A, B, C, D, E [2] 🟩 Adjacency Matrix: Neighbors ↳ Add 1 for each edge to neighbors ↳ Repeat in both directions (e.g., A->C, C->A) ↳ Repeat for both GCN layers [3] 🟩 Adjacency Matrix: Self ↳ Add 1's for each self loop ↳ Equivalent to adding the identity matrix ↳ Repeat for both GCN layers [4] 🟪 GCN1: Messages ↳ Multiply the node embeddings 🟨 with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is one message per node [5] 🟪 GCN1: Pooling ↳ Multiply the messages with the adjacent matrix ↳ The purpose is the pool messages from each node's neighbors as well as from the node itself. ↳ The result is a new feature per node [6] 🟪 GCN1: Visualize ↳ For node 1, visualize how messages are pooled to obtain a new feature for better understanding ↳ [3,0,1] + [1,0,0] = [4,0,1] [7] 🟪 GCN2: Messages ↳ Multiply the node features with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is one message per node [8] 🟪 GCN2: Pooling ↳ Multiply the messages with the adjacent matrix ↳ The result is a new feature per node [9] 🟪 GCN2: Visualize ↳ For node 3, visualize how messages are pooled to obtain a new feature for better understanding ↳ [1,2,4] + [1,3,5] + [0,0,1] = [2,5,10] [10] 🟦 FCN: Linear 1 + ReLU ↳ Multiply node features with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is a new feature per node ↳ Unlike in GCN layers, no messages from other nodes are included. [11] 🟦 FCN: Linear 2 ↳ Multiply node features with weights and biases [12] 🟦 FCN: Sigmoid ↳ Apply the Sigmoid activation function ↳ The purpose is to obtain a probability value for each node ↳ One way to calculate Sigmoid by hand ✍️ is to use the approximation below: • >= 3 → 1 • 0 → 0.5 • <= -3 → 0 -- Outputs -- A: 0 (Very unlikely) B: 1 (Very likely) C: 1 (Very likely) D: 1 (Very likely) E: 0.5 (Neutral)

[Graph Convolutional Network] by hand ✍️ Graph Convolutional Networks (GCNs), introduced by Thomas Kipf and Max Welling in 2017, have emerged as a powerful tool in the analysis and interpretation of data structured as graphs. This exercise demonstrates how GCN works in a simple application: binary classification. -- Goal -- Predict if a node in a graph is X. -- Architecture -- 🟪 Graph Convolutional Network (GCN) 1. GCN1(4,3) 2. GCN2(3,3) 🟦 Fully Connected Network (FCN) 1. Linear1(3,5) 2. ReLU 3. Linear2(5,1) 4. Sigmoid Simplications: • Adjacent matrices are not normalized. • ReLU is applied to messages directly. -- Walkthrough -- [1] Given ↳ A graph with five nodes A, B, C, D, E [2] 🟩 Adjacency Matrix: Neighbors ↳ Add 1 for each edge to neighbors ↳ Repeat in both directions (e.g., A->C, C->A) ↳ Repeat for both GCN layers [3] 🟩 Adjacency Matrix: Self ↳ Add 1's for each self loop ↳ Equivalent to adding the identity matrix ↳ Repeat for both GCN layers [4] 🟪 GCN1: Messages ↳ Multiply the node embeddings 🟨 with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is one message per node [5] 🟪 GCN1: Pooling ↳ Multiply the messages with the adjacent matrix ↳ The purpose is the pool messages from each node's neighbors as well as from the node itself. ↳ The result is a new feature per node [6] 🟪 GCN1: Visualize ↳ For node 1, visualize how messages are pooled to obtain a new feature for better understanding ↳ [3,0,1] + [1,0,0] = [4,0,1] [7] 🟪 GCN2: Messages ↳ Multiply the node features with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is one message per node [8] 🟪 GCN2: Pooling ↳ Multiply the messages with the adjacent matrix ↳ The result is a new feature per node [9] 🟪 GCN2: Visualize ↳ For node 3, visualize how messages are pooled to obtain a new feature for better understanding ↳ [1,2,4] + [1,3,5] + [0,0,1] = [2,5,10] [10] 🟦 FCN: Linear 1 + ReLU ↳ Multiply node features with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is a new feature per node ↳ Unlike in GCN layers, no messages from other nodes are included. [11] 🟦 FCN: Linear 2 ↳ Multiply node features with weights and biases [12] 🟦 FCN: Sigmoid ↳ Apply the Sigmoid activation function ↳ The purpose is to obtain a probability value for each node ↳ One way to calculate Sigmoid by hand ✍️ is to use the approximation below: • >= 3 → 1 • 0 → 0.5 • <= -3 → 0 -- Outputs -- A: 0 (Very unlikely) B: 1 (Very likely) C: 1 (Very likely) D: 1 (Very likely) E: 0.5 (Neutral)

Tom Yeh

46,779 Aufrufe • vor 2 Jahren

👉🏻Dribble Tag👈🏻 An activity that is accessible and fun for a broad range of ages. 📈 2 students, 4 cones, 1 ball. Scaffold the activity: L.1 - no ball L.2 - player evading has a ball L.3 - both players have a ball Please retweet if useful 🙏

👉🏻Dribble Tag👈🏻 An activity that is accessible and fun for a broad range of ages. 📈 2 students, 4 cones, 1 ball. Scaffold the activity: L.1 - no ball L.2 - player evading has a ball L.3 - both players have a ball Please retweet if useful 🙏

Blueprint Aquatics

22,841 Aufrufe • vor 3 Jahren

In my class, I teach the autoencoder by asking everyone to stand up. 🙆 Stretch your arms out wide. Imagine you are holding a heavy textbook (like Introduction to Algorithms by Prof. Cormen), the whole thing, every page. Now bring your hands slowly together until they almost touch your neck. The bottle "neck." The final exam is tomorrow and you are allowed one cheat sheet: whatever you can scribble on your palm. All nine hundred pages have to survive the squeeze. That is the encoder. Now imagine you sit in the exam. Push your arms back out to where they started. You try to rebuild the textbook from your palm notes. That is the decoder. Of course you cannot get every page back. What you get back is what mattered enough to write down, and the gap between the two is the loss the network is trying to shrink. I call it AI by Arms 🙆. It gets a laugh, and then it gets remembered. Goal: squeeze four numbers down to two, then rebuild the original four from them. 1. Given Let us start with four training examples: X1, X2, X3, X4. 2. Auto (copy to targets) We copy the training examples straight into the targets. That is the whole trick behind the name: "auto" is Greek for "self", and the data is its own label. 3. Encoder, layer 1 Let us multiply the inputs by the weights, add the biases, and apply ReLU. Negative values get crossed out and become zero. 4. Encoder, layer 2 (the bottleneck) We do it again, and now the four dimensions have become two. This layer is called the bottleneck, because everything has to fit through it. 5. Decoder, layer 1 Let us go back the other way: multiply, add, ReLU. This time there are no negatives to cross out. 6. Decoder, layer 2 We multiply once more and get the outputs Y. This is the decoder's attempt to rebuild the four original numbers from the two it was given. 7. MSE loss gradients Let us compare Y with the targets Y'. The gradient is 2 x (Y - Y'): subtract, then double. Those gradients kick off backpropagation, and the weights start to learn. Your entire education is all about encoding and decoding!

In my class, I teach the autoencoder by asking everyone to stand up. 🙆 Stretch your arms out wide. Imagine you are holding a heavy textbook (like Introduction to Algorithms by Prof. Cormen), the whole thing, every page. Now bring your hands slowly together until they almost touch your neck. The bottle "neck." The final exam is tomorrow and you are allowed one cheat sheet: whatever you can scribble on your palm. All nine hundred pages have to survive the squeeze. That is the encoder. Now imagine you sit in the exam. Push your arms back out to where they started. You try to rebuild the textbook from your palm notes. That is the decoder. Of course you cannot get every page back. What you get back is what mattered enough to write down, and the gap between the two is the loss the network is trying to shrink. I call it AI by Arms 🙆. It gets a laugh, and then it gets remembered. Goal: squeeze four numbers down to two, then rebuild the original four from them. 1. Given Let us start with four training examples: X1, X2, X3, X4. 2. Auto (copy to targets) We copy the training examples straight into the targets. That is the whole trick behind the name: "auto" is Greek for "self", and the data is its own label. 3. Encoder, layer 1 Let us multiply the inputs by the weights, add the biases, and apply ReLU. Negative values get crossed out and become zero. 4. Encoder, layer 2 (the bottleneck) We do it again, and now the four dimensions have become two. This layer is called the bottleneck, because everything has to fit through it. 5. Decoder, layer 1 Let us go back the other way: multiply, add, ReLU. This time there are no negatives to cross out. 6. Decoder, layer 2 We multiply once more and get the outputs Y. This is the decoder's attempt to rebuild the four original numbers from the two it was given. 7. MSE loss gradients Let us compare Y with the targets Y'. The gradient is 2 x (Y - Y'): subtract, then double. Those gradients kick off backpropagation, and the weights start to learn. Your entire education is all about encoding and decoding!

Tom Yeh

22,749 Aufrufe • vor 3 Monaten

Prism, 328 chars #つぶやきGLSL for(L*=I;I++<80.;L+=pow(.0024/length(P* mat2(cos(10./exp(R/.1)*pow(sin(A/8.), 3.)+A/8.+vec4(0,33,11,0)))+R* sin(I*.5+A/4.)*smoothstep(0., 1.,(.95 + .05 * sin(A)))*vec2(cos(I*.2), sin(I*.2 ))) * (1. + cos(I*.1+R*4.-A-A + vec4(0,1,2,0))), L-L+1.1));

Prism, 328 chars #つぶやきGLSL for(L=I;I++<80.;L+=pow(.0024/length(P mat2(cos(10./exp(R/.1)pow(sin(A/8.), 3.)+A/8.+vec4(0,33,11,0)))+R sin(I.5+A/4.)smoothstep(0., 1.,(.95 + .05 * sin(A)))vec2(cos(I.2), sin(I.2 ))) (1. + cos(I.1+R4.-A-A + vec4(0,1,2,0))), L-L+1.1));

kishimisu

45,116 Aufrufe • vor 2 Jahren

HOW did we let these two get away with this last year? The LARRIEST move in '22, 4 PARTS 1. '08? L played Danny 2. Oct 18 L mentions he still has the jacket! 3. Oct 31 H plays Danny Zuko(/Sandy), sings Hopelessly Devoted 4. Dec 8- L says his fave Grease song is Hopelessly Devoted

HOW did we let these two get away with this last year? The LARRIEST move in '22, 4 PARTS 1. '08? L played Danny 2. Oct 18 L mentions he still has the jacket! 3. Oct 31 H plays Danny Zuko(/Sandy), sings Hopelessly Devoted 4. Dec 8- L says his fave Grease song is Hopelessly Devoted

✨Megs✨🔮🐓

14,396 Aufrufe • vor 3 Jahren

Event 2 vec2 p=(FC.xy*2.-r)/r.y/.3,v;for(float i,l,f;i++<9.;o+=.1/abs(l=dot(p,p)-5.-2./v.y)*(cos(i/3.+.1/l+vec4(1,2,3,4))+1.))for(v=p,f=0.;f++<9.;v+=sin(ceil(v.yx*f+i*.3)+r-t/2.)/f);o=max(tanh(o+(o=texture(b,(FC.xy+r.y*.04*sin(FC.xy+FC.yx/.6))/r))*o),.0);

Event 2 vec2 p=(FC.xy2.-r)/r.y/.3,v;for(float i,l,f;i++<9.;o+=.1/abs(l=dot(p,p)-5.-2./v.y)(cos(i/3.+.1/l+vec4(1,2,3,4))+1.))for(v=p,f=0.;f++<9.;v+=sin(ceil(v.yxf+i.3)+r-t/2.)/f);o=max(tanh(o+(o=texture(b,(FC.xy+r.y.04sin(FC.xy+FC.yx/.6))/r))*o),.0);

Xor

41,931 Aufrufe • vor 9 Monaten

This looks l like The Islands estate in Harties 😍🫶🏾 1 day is 1 day 🤞🏿

This looks l like The Islands estate in Harties 😍🫶🏾 1 day is 1 day 🤞🏿

❎e k ℹ️

127,602 Aufrufe • vor 3 Jahren

$TSLA 🇰🇷 The Model 3 RWD and Model Y L sold in Korea are cheaper than any other country. • Model 3 RWD ($28,000~) • Model Y L ($43,000~)

$TSLA 🇰🇷 The Model 3 RWD and Model Y L sold in Korea are cheaper than any other country. • Model 3 RWD ($28,000~) • Model Y L ($43,000~)

Tsla Chan

17,263 Aufrufe • vor 7 Monaten

Observer vec2 p=(FC.xy*2.-r)/r.y/.2,v;for(float i,l,f;i++<1e1;o+=.03/max(l=length(v)-i,-l*3.)*(cos(t-i*.4+.1/l+vec4(0,1,2,3))+1.1))for(v=p,f=0.;f++<9.;v+=sin(ceil(v*f+i*.9)-t/2.)/f);o=max(tanh(o+(o=texture(b,(FC.xy+r.y*.05*sin(FC.xy+FC.yx/.6))/r))*o),.0);

Observer vec2 p=(FC.xy2.-r)/r.y/.2,v;for(float i,l,f;i++<1e1;o+=.03/max(l=length(v)-i,-l3.)(cos(t-i.4+.1/l+vec4(0,1,2,3))+1.1))for(v=p,f=0.;f++<9.;v+=sin(ceil(vf+i.9)-t/2.)/f);o=max(tanh(o+(o=texture(b,(FC.xy+r.y.05sin(FC.xy+FC.yx/.6))/r))*o),.0);

Xor

10,558 Aufrufe • vor 9 Monaten

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features across positions, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens across feature dimensions, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes across positions, so a feature can see its neighbours. The FFN mixes across feature dimensions, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning

Tom Yeh

26,279 Aufrufe • vor 3 Monaten

Regular RORE to BRP SIERRA MADRE this morning. BRP CABRA, Unaizah Mae 1, and M/L Kalayaan water cannoned by China Coast Guard. M/L Kalayaan suffered serious engine damage. Contrary to China Coast Guard disinformation, UM1 rammed by CCG vessel.

Regular RORE to BRP SIERRA MADRE this morning. BRP CABRA, Unaizah Mae 1, and M/L Kalayaan water cannoned by China Coast Guard. M/L Kalayaan suffered serious engine damage. Contrary to China Coast Guard disinformation, UM1 rammed by CCG vessel.

Jay Tarriela

215,555 Aufrufe • vor 2 Jahren

Black Tie l OUT NOW l Wear your black tie,and you will be invited to ‘The party of silence’. #BlackTieJeffSatur

Black Tie l OUT NOW l Wear your black tie,and you will be invited to ‘The party of silence’. #BlackTieJeffSatur

𝗝 𝗘 𝗙 𝗙 𝗦 𝗔 𝗧 𝗨 𝗥 (罗杰夫)

546,153 Aufrufe • vor 3 Jahren

To label the LV walls in TTE, use the same mnemonics (A-L-P-I-S) but read the letters clockwise 🔁. If you don’t like A-L-P-I-S, delete the letter “P” and make L-I-S-A instead #echofirst

To label the LV walls in TTE, use the same mnemonics (A-L-P-I-S) but read the letters clockwise 🔁. If you don’t like A-L-P-I-S, delete the letter “P” and make L-I-S-A instead #echofirst

Alexander Mladenow MD

13,946 Aufrufe • vor 10 Monaten

[SVM] by Hand ✍️ Support Vector Machines (SVMs) reigned supreme in machine learning before the ascendancy of the deep learning revolution. This exercise compares Linear vs RBF SVMs---how they classify two test vectors, after SVMs are trained from six training vectors. [1] Given xi: Six training vectors (blue rows 🟦) yi: Labels Using xi and yi, we learned ai and b (red borders): ↳ai: coefficient for each training vector i. • Non-zero: A Support Vector that defines the decision boundary • Zero: Too far from the decision boundary, ignored ↳b: bias (how much the decision boundary should be shifted) x’j: Two test vectors (yellow columns 🟨) (To simplify hand calculation, training and test vectors are not normalized.) ~ 𝗟𝗶𝗻𝗲𝗮𝗿 𝗦𝗩𝗠 ~ 🟩 Kernel Matrix (K) [2]-[3] [2] Test Vector 1 ↳ Take dot product between the test vector 🟨 and every training vector 🟦 ↳ The dot product approximates the “cosine similarity” between two vectors ↳ Output: 1st column of K [3] Test Vector 2 ↳ Similar to [2] ↳ Output: 2nd column of K 🟧 Decision Boundary [4]-[6] [4] Unsigned Coefficients → Signed Weights ↳ Multiply each coefficient with the corresponding label ↳ The 2nd training vector is NOT a support vector because its coefficient is 0. [5] Weighted Combination ↳ Multiply weights and bias with K ↳ Output: “signed” distance to the decision boundary ... X'1: (2)*13+0+(1)*12+(-1)*15+(1)*19+(-1)*27+(2) = 17 X'2: (2)*2+0+(1)*3+(-1)*4+(1)*8+(-1)*8+(2) = 5 ... [6] Classify ↳ Take the sign ... X'1: 1 > 0 → Positive + X'2: 5 > 0 → Positive + ... ~ 𝗥𝗕𝗙 𝗦𝗩𝗠 ~ Given ai: Learned coefficients b: Learned bias 🟩 Kernel Matrix (K) [7]-[15] 𝘛𝘦𝘴𝘵 𝘝𝘦𝘤𝘵𝘰𝘳 (𝘟’1) 🟨 L2 Distance 🟪 [7]-[9] [7] Squared Difference ... i=1: (1-2)^2=1, (2-4)^2=4, (1-3)^2=4 ... [8] Sum [9] Square Root [10] Negative Scaling ↳ Multiply by -1: Note that L2 is a distance metric. The negation converts distance to similarity. ↳ Multiply by gamma γ: The purpose is to control how much influence each training example has. A small gamma means each training example pulls the decision boundary more lightly, resulting in smoother decision boundaries. ↳ The result is “negative scaled L2” [11] Exponentiate ↳ Raise e to the power of the “negative scaled L2” ↳ Use the provided table to look up the value of e^ ↳ Output: The 1st column of K 𝘛𝘦𝘴𝘵 𝘝𝘦𝘤𝘵𝘰𝘳 2 (𝘟’2) 🟨 L2 Distance 🟪 [12]-[14] [12] Squared Difference [13] Sum [14] Square Root [15] Negative Scaling [16] Exponentiate ↳Output: The 2nd column of K 🟧 Decision Boundary [17]-[19] [17] Unsigned Coefficients → Signed Weights [18] Weighted Combination ... X'1: (20)*0.7+(-10)*0.8+(10)*0.9+(-10)*0.7+0+(-10)*0.9+(-1) = -2 X'2: (20)*0.9+(-10)*0.7+(10)*0.8+(-10)*0.8+0+(-10)*0.9+(-1) = 1 ... [19] Classify ... X'1: -2 0 → + ... ↳Output: The 2nd column of K 🟧 Decision Boundary [17]-[19] [17] Unsigned Coefficients → Signed Weights [18] Weighted Combination ... X'1: (20)*0.7+(-10)*0.8+(10)*0.9+(-10)*0.7+0+(-10)*0.9+(-1) = -2 X'2: (20)*0.9+(-10)*0.7+(10)*0.8+(-10)*0.8+0+(-10)*0.9+(-1) = 1 ... [19] Classify ... X'1: -2 0 → + ...

[SVM] by Hand ✍️ Support Vector Machines (SVMs) reigned supreme in machine learning before the ascendancy of the deep learning revolution. This exercise compares Linear vs RBF SVMs---how they classify two test vectors, after SVMs are trained from six training vectors. [1] Given xi: Six training vectors (blue rows 🟦) yi: Labels Using xi and yi, we learned ai and b (red borders): ↳ai: coefficient for each training vector i. • Non-zero: A Support Vector that defines the decision boundary • Zero: Too far from the decision boundary, ignored ↳b: bias (how much the decision boundary should be shifted) x’j: Two test vectors (yellow columns 🟨) (To simplify hand calculation, training and test vectors are not normalized.) ~ 𝗟𝗶𝗻𝗲𝗮𝗿 𝗦𝗩𝗠 ~ 🟩 Kernel Matrix (K) [2]-[3] [2] Test Vector 1 ↳ Take dot product between the test vector 🟨 and every training vector 🟦 ↳ The dot product approximates the “cosine similarity” between two vectors ↳ Output: 1st column of K [3] Test Vector 2 ↳ Similar to [2] ↳ Output: 2nd column of K 🟧 Decision Boundary [4]-[6] [4] Unsigned Coefficients → Signed Weights ↳ Multiply each coefficient with the corresponding label ↳ The 2nd training vector is NOT a support vector because its coefficient is 0. [5] Weighted Combination ↳ Multiply weights and bias with K ↳ Output: “signed” distance to the decision boundary ... X'1: (2)13+0+(1)12+(-1)15+(1)19+(-1)27+(2) = 17 X'2: (2)2+0+(1)3+(-1)4+(1)8+(-1)8+(2) = 5 ... [6] Classify ↳ Take the sign ... X'1: 1 > 0 → Positive + X'2: 5 > 0 → Positive + ... ~ 𝗥𝗕𝗙 𝗦𝗩𝗠 ~ Given ai: Learned coefficients b: Learned bias 🟩 Kernel Matrix (K) [7]-[15] 𝘛𝘦𝘴𝘵 𝘝𝘦𝘤𝘵𝘰𝘳 (𝘟’1) 🟨 L2 Distance 🟪 [7]-[9] [7] Squared Difference ... i=1: (1-2)^2=1, (2-4)^2=4, (1-3)^2=4 ... [8] Sum [9] Square Root [10] Negative Scaling ↳ Multiply by -1: Note that L2 is a distance metric. The negation converts distance to similarity. ↳ Multiply by gamma γ: The purpose is to control how much influence each training example has. A small gamma means each training example pulls the decision boundary more lightly, resulting in smoother decision boundaries. ↳ The result is “negative scaled L2” [11] Exponentiate ↳ Raise e to the power of the “negative scaled L2” ↳ Use the provided table to look up the value of e^ ↳ Output: The 1st column of K 𝘛𝘦𝘴𝘵 𝘝𝘦𝘤𝘵𝘰𝘳 2 (𝘟’2) 🟨 L2 Distance 🟪 [12]-[14] [12] Squared Difference [13] Sum [14] Square Root [15] Negative Scaling [16] Exponentiate ↳Output: The 2nd column of K 🟧 Decision Boundary [17]-[19] [17] Unsigned Coefficients → Signed Weights [18] Weighted Combination ... X'1: (20)0.7+(-10)0.8+(10)0.9+(-10)0.7+0+(-10)0.9+(-1) = -2 X'2: (20)0.9+(-10)0.7+(10)0.8+(-10)0.8+0+(-10)0.9+(-1) = 1 ... [19] Classify ... X'1: -2 0 → + ... ↳Output: The 2nd column of K 🟧 Decision Boundary [17]-[19] [17] Unsigned Coefficients → Signed Weights [18] Weighted Combination ... X'1: (20)0.7+(-10)0.8+(10)0.9+(-10)0.7+0+(-10)0.9+(-1) = -2 X'2: (20)0.9+(-10)0.7+(10)0.8+(-10)0.8+0+(-10)0.9+(-1) = 1 ... [19] Classify ... X'1: -2 0 → + ...

Tom Yeh

35,708 Aufrufe • vor 2 Jahren