[Backpropagation] by Hand✍️ [1] Forward Pass ↳ Given a... multi layer perceptron (3 levels), an input vector X, predictions Y^{Pred} = [0.5, 0.5, 0], and ground truth label Y^{Target} = [0, 1, 0]. [2] Backpropagation ↳ Insert cells to hold our calculations. [3] Layer 3 - Softmax (blue) ↳ Calculate ∂L / ∂z3 directly using the simple equation: Y^{Pred} - Y^{Target} = [0.5, -0.5, 0]. ↳ This simple equation is the benefit of using Softmax and Cross Entropy Loss together. [4] Layer 3 - Weights (orange) & Biases (black) ↳ Calculate ∂L / ∂W3 and ∂L / ∂b3 by multiplying ∂L / ∂z3 and [ a2 | 1 ]. [5] Layer 2 - Activations (green) ↳ Calculate ∂L / ∂a2 by multiplying ∂L / ∂z3 and W3. [6] Layer 2 - ReLU (blue) ↳ Calculate ∂L / ∂z2 by multiplying ∂L / ∂a2 with 1 for positive values and 0 otherwise. [7] Layer 2 - Weights (orange) & Biases (black) ↳ Calculate ∂L / ∂W2 and ∂L / ∂b2 by multiplying ∂L / ∂z2 and [ a1 | 1 ]. [8] Layer 1 - Activations (green) ↳ Calculate ∂L / ∂a1 by multiplying ∂L / ∂z2 and W2. [9] Layer 1 - ReLU (blue) ↳ Calculate ∂L / ∂z1 by multiplying ∂L / ∂a1 with 1 for positive values and 0 otherwise. [10] Layer 1 - Weights (orange) & Biases (black) ↳ Calculate ∂L / ∂W1 and ∂L / ∂b1 by multiplying ∂L / ∂z1 and [ x | 1 ]. [11] Gradient Descent ↳ Update weights and biases (typically a learning rate is applied here). 💡 Matrix Multiplication is All You Need: Just like in the forward pass, backpropagation is all about matrix multiplications. You can definitely do everything by hand as I demonstrated in this exercise, albeit slow and imperfect. This is why GPU's ability to multiply matrices efficiently plays such an important role in the deep learning evolution. This is why NVIDIA is now close to $1 trillion in valuation. 💡Exploding Gradients: We can already see the gradients are getting larger as we back-propagate up, even in this simple 3-layer network. This motivates using methods like skip connections to handle exploding (or diminishing) gradients as in the ResNet. I did the calculations entirely by hand. Please let me know if you spot any error or have any questions!show more

[Graph Convolutional Network] by hand ✍️ Graph Convolutional Networks... (GCNs), introduced by Thomas Kipf and Max Welling in 2017, have emerged as a powerful tool in the analysis and interpretation of data structured as graphs. This exercise demonstrates how GCN works in a simple application: binary classification. -- Goal -- Predict if a node in a graph is X. -- Architecture -- 🟪 Graph Convolutional Network (GCN) 1. GCN1(4,3) 2. GCN2(3,3) 🟦 Fully Connected Network (FCN) 1. Linear1(3,5) 2. ReLU 3. Linear2(5,1) 4. Sigmoid Simplications: • Adjacent matrices are not normalized. • ReLU is applied to messages directly. -- Walkthrough -- [1] Given ↳ A graph with five nodes A, B, C, D, E [2] 🟩 Adjacency Matrix: Neighbors ↳ Add 1 for each edge to neighbors ↳ Repeat in both directions (e.g., A->C, C->A) ↳ Repeat for both GCN layers [3] 🟩 Adjacency Matrix: Self ↳ Add 1's for each self loop ↳ Equivalent to adding the identity matrix ↳ Repeat for both GCN layers [4] 🟪 GCN1: Messages ↳ Multiply the node embeddings 🟨 with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is one message per node [5] 🟪 GCN1: Pooling ↳ Multiply the messages with the adjacent matrix ↳ The purpose is the pool messages from each node's neighbors as well as from the node itself. ↳ The result is a new feature per node [6] 🟪 GCN1: Visualize ↳ For node 1, visualize how messages are pooled to obtain a new feature for better understanding ↳ [3,0,1] + [1,0,0] = [4,0,1] [7] 🟪 GCN2: Messages ↳ Multiply the node features with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is one message per node [8] 🟪 GCN2: Pooling ↳ Multiply the messages with the adjacent matrix ↳ The result is a new feature per node [9] 🟪 GCN2: Visualize ↳ For node 3, visualize how messages are pooled to obtain a new feature for better understanding ↳ [1,2,4] + [1,3,5] + [0,0,1] = [2,5,10] [10] 🟦 FCN: Linear 1 + ReLU ↳ Multiply node features with weights and biases ↳ Apply ReLU (negatives → 0) ↳ The result is a new feature per node ↳ Unlike in GCN layers, no messages from other nodes are included. [11] 🟦 FCN: Linear 2 ↳ Multiply node features with weights and biases [12] 🟦 FCN: Sigmoid ↳ Apply the Sigmoid activation function ↳ The purpose is to obtain a probability value for each node ↳ One way to calculate Sigmoid by hand ✍️ is to use the approximation below: • >= 3 → 1 • 0 → 0.5 • <= -3 → 0 -- Outputs -- A: 0 (Very unlikely) B: 1 (Very likely) C: 1 (Very likely) D: 1 (Very likely) E: 0.5 (Neutral)show more

👉🏻Dribble Tag👈🏻 An activity that is accessible and fun... for a broad range of ages. 📈 2 students, 4 cones, 1 ball. Scaffold the activity: L.1 - no ball L.2 - player evading has a ball L.3 - both players have a ball Please retweet if useful 🙏show more

Prism, 328 chars #つぶやきGLSL for(L*=I;I++&lt;80.;L+=pow(.0024/length(P* mat2(cos(10./exp(R/.1)*pow(sin(A/8.), 3.)+A/8.+vec4(0,33,11,0)))+R* sin(I*.5+A/4.)*smoothstep(0., 1.,(.95... + .05 * sin(A)))*vec2(cos(I*.2), sin(I*.2 ))) * (1. + cos(I*.1+R*4.-A-A + vec4(0,1,2,0))), L-L+1.1));show more

HOW did we let these two get away with... this last year? The LARRIEST move in '22, 4 PARTS 1. '08? L played Danny 2. Oct 18 L mentions he still has the jacket! 3. Oct 31 H plays Danny Zuko(/Sandy), sings Hopelessly Devoted 4. Dec 8- L says his fave Grease song is Hopelessly Devotedshow more

Event 2 vec2 p=(FC.xy*2.-r)/r.y/.3,v;for(float i,l,f;i++&lt;9.;o+=.1/abs(l=dot(p,p)-5.-2./v.y)*(cos(i/3.+.1/l+vec4(1,2,3,4))+1.))for(v=p,f=0.;f++&lt;9.;v+=sin(ceil(v.yx*f+i*.3)+r-t/2.)/f);o=max(tanh(o+(o=texture(b,(FC.xy+r.y*.04*sin(FC.xy+FC.yx/.6))/r))*o),.0);

$TSLA 🇰🇷 The Model 3 RWD and Model Y... L sold in Korea are cheaper than any other country. • Model 3 RWD ($28,000~) • Model Y L ($43,000~)show more

Observer vec2 p=(FC.xy*2.-r)/r.y/.2,v;for(float i,l,f;i++&lt;1e1;o+=.03/max(l=length(v)-i,-l*3.)*(cos(t-i*.4+.1/l+vec4(0,1,2,3))+1.1))for(v=p,f=0.;f++&lt;9.;v+=sin(ceil(v*f+i*.9)-t/2.)/f);o=max(tanh(o+(o=texture(b,(FC.xy+r.y*.05*sin(FC.xy+FC.yx/.6))/r))*o),.0);

Regular RORE to BRP SIERRA MADRE this morning. BRP... CABRA, Unaizah Mae 1, and M/L Kalayaan water cannoned by China Coast Guard. M/L Kalayaan suffered serious engine damage. Contrary to China Coast Guard disinformation, UM1 rammed by CCG vessel.show more

Black Tie l OUT NOW l Wear your black... tie,and you will be invited to ‘The party of silence’. #BlackTieJeffSaturshow more

Samantha from Her was talking to just 8,316 people... ChatGPT is talking to 7 0 0 M I L L I O N Her was set in... 2025show more

plave writing plave and plli with the sparklers 🥺... P L A V E P L L I 🤍show more

all the parameters (weights and biases) of the same... multilayer perception over 200 epochs red/blue/yellow-three layers The top right knob in each layer being the weightshow more

To label the LV walls in TTE, use the... same mnemonics (A-L-P-I-S) but read the letters clockwise 🔁. If you don’t like A-L-P-I-S, delete the letter “P” and make L-I-S-A instead #echofirstshow more

LSTM by hand✍️Excel ~ I designed this exercise to... show it is possible to calculate a simple LSTM by hand. Green🟩: Short Term. Blue🟦: Long Term. +equations. +medium.👇Join the 'AI Math' community. Download xlsx.show more

Did you know? Killer Instinct on SNES was sold... in a special black cartridge and included a "Killer Cuts" CD. You can play as the boss Eyedol by selecting Cinder and quickly tapping Right + L, R, X, B, Y, A at the select screen! Combo by Dark Noobshow more

All sides of the cube are squares (save this... info) fgh shows in the first puzzle (with 0 and 1) + “From.EXO PLANET” with F and E in bold + connection L: it actually starts in H: h > g > f > l > e A to C but K and H in hidden: e > a > k > b > c > d> h view from the front: if we move the cube, the sides are not a square anymore, there’s always a line missing. They’re free to pass beginning = end = it starts and end in hshow more

Expanding $LAI Utility: Gas for Layer-2 Testnet, Mainnet &... Ecosystem Apps 🧬 We are excited to unveil new additions to the existing $LAI utilities, in line with the development of our ZK Layer-2 network. (1) $LAI integrated on the ZK Layer-2 Testnet as transaction gas (2) $LAI integrated on the ZK Layer-2 Mainnet as transaction gas (3) $LAI adopted by all ecosystem apps deploying on our network 👉 Explore the updated $LAI Economy Page: 👉 Dive into Gasless Testnet:show more

💡Back to basics! What is a Layer 1 blockchain?... It’s the base layer of a network — the foundation where all transactions, smart contracts, and dApps run. Think of it as the “operating system” of Web3. And just like any OS, its speed and security define everything built on top. 👉 Meet CrossFi Blockchain: fast, secure, and built to scale.show more