AlphaFold by hand✍️ Excel ~ I designed this exercise... to show (1) MSA multi-head attention, (2) Pair triangular update, two key components of the EvoFormer architecture.👇Join the AI Math community. Download xlsx.show more

Transformer by hand✍️ Excel ~ I designed this exercise... to show the core math of a Transformer model is to combine columns (attention), combine rows (feed forward), and repeat.👇Join the 'AI Math' community. 👇Download xlsx.show more

LSTM by hand✍️Excel ~ I designed this exercise to... show it is possible to calculate a simple LSTM by hand. Green🟩: Short Term. Blue🟦: Long Term. +equations. +medium.👇Join the 'AI Math' community. Download xlsx.show more

Autoencoder by hand✍️Excel~ I designed this exercise to show... how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. 👇Join the 'AI Math' community. Download xlsx.show more

ResNet by hand✍️Excel~ I designed this exercise to compare... a ResNet to an MLP and show skip-connections are simply identity matrices next to weights and biases. I also made a medium version to show how it scales. 👇Join the 'AI Math' community. Download xlsx.show more

Autoencoder by hand✍️Excel~ I designed this exercise to show... how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. I also made a medium version.👇Join the 'AI Math' community. Download xlsx.show more

Backpropagation by hand✍️ ~ spreadsheet. I designed this exercise... to show it is possible to calculate backpropagation for a non-trivial, three-layer network by hand. p.s. I just started this community to share useful resources on AI math. 👇 Join the community.show more

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~... I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.show more

At MIT, the only course I ever dropped was... signal processing. The DFT math was too intimidating. It’s so easy to just type fft() in MATLAB and move on. Years later, I finally did DFT by hand. ✍️ If you are also afraid of DFT, I hope this helps! ⬇️ Download:show more

At MIT, I learned about RNNs in my NLP... class with Prof. Michael Collins. He built a model from my keystrokes to predict who I was. To me, it felt like a magic box. Years later, when I had to teach RNNs, I forced myself to go inside the box. ⬇️ Download: First, with a tiny example on paper by hand ✍️. Then, a slightly larger one in Excel. That’s when it finally clicked: 👉 The weights are reused (weight matrices on the left side) 👉 The hidden states are passed down (H's) When I built it by hand and saw everything visually, it clicked, just math you can actually trace. Now I try to give others the same “aha!” moment I had. ⬇️ Download Excel:show more

Single vs Multi-hand Attention by hand ✍️ Resize matrices... yourself 👉 The most important fact about multi-head attention: it has the same parameter count as single-head attention. The difference is purely structural — same total Wqkv weights, partitioned into smaller q–k–v triples. Look at the two diagrams below. Both Wqkv matrices have the same height — same number of weight rows, same number of parameters. What changes is how that single tall block is sliced. • Left. One head. The full Wqkv produces one big QKV: a tall Q (36 rows), a tall K, a tall V. One scoring computation runs over those full-width tensors. • Right. 3 heads. The same-height Wqkv is sliced into 3 smaller q–k–v triples — each 12 rows tall. 3 scoring computations run in parallel, each a thinner version of the left. The compute trade-off — kind of. Same Wqkv weights. Multi-head runs the attention scoring S = Kᵀ × Q once per head, so the dot-product count multiplies by H. • Single-head: seq × seq = 40² = 1600 dot products • Multi-head: seq × seq × H = 40² × 3 = 4800 dot products (3×) But each multi-head dot product is narrower — its inner dimension is head_dim instead of H × head_dim. So when you count actual scalar multiplications, the totals are equal: • Single-head: seq² × (H × head_dim) = 40² × 36 = 57600 • Multi-head: seq² × H × head_dim = 40² × 3 × 12 = 57600 Same FLOPs. Multi-head buys you H independent attention patterns at no extra weight cost and no extra arithmetic cost — it's the same total compute, sliced into H finer-grained heads.show more

🚨 Omg.. the $1M AI War just began. Genspark... launched a $1 MILLION Side-by-Side AI Showdown, challenging any AI, including ChatGPT. I tested many of them on the same tasks… but In every test, Genspark’s results came out on top. Check the head-to-head below: 👇show more

HOLY CRAP! I can't tell you how big this... is for the medical community and drug discovery: Google Announces AlphaFold 3 AI. Details: Enhanced Molecular Prediction: AlphaFold 3 predicts the structure and interactions of all life's molecules, including proteins, DNA, RNA, ligands, and more, with unprecedented accuracy. Improved Interaction Accuracy: For protein interactions with other molecule types, AlphaFold 3 offers at least a 50% improvement over existing methods, and doubles the accuracy for some critical interactions. Transformative Potential for Science and Medicine: The model aims to deepen our understanding of biological processes and significantly advance drug discovery efforts. Accessibility for Researchers: AlphaFold 3's capabilities are largely accessible for free via the AlphaFold Server, providing an essential tool for scientific research. Drug Design Innovation: AlphaFold 3 is utilized by Isomorphic Labs in collaboration with pharmaceutical companies to accelerate drug design, potentially leading to new treatments for various diseases. Foundation in AlphaFold 2: Building on the breakthroughs of AlphaFold 2, this version extends its scope beyond proteins to a wide range of biomolecules, enhancing its utility in scientific research and application. Global Accessibility and Educational Support: The AlphaFold Server is a free platform for non-commercial research worldwide, supported by educational resources to foster wider adoption and innovation. Empowering Rapid Scientific Advancements: By making detailed molecular interactions easily accessible, AlphaFold 3 enables faster hypothesis testing and could reduce the time and cost typically associated with experimental protein-structure prediction. Responsible Development and Deployment: DeepMind has engaged with domain experts to assess the impacts and potential risks of AlphaFold, ensuring its responsible use in the scientific community. Broad Implications for Biology:AlphaFold 3 helps reveal complex cellular mechanisms and interactions, offering insights that could lead to improved agricultural crops, enhanced understanding of diseases, and novel therapeutic strategies.show more

Calling All AI Filmmakers💀🎃 Since I didn't have time... to make my annual AI Halloween short film this year I figured I'd enlist the help of the AI community. Help me finish ENTER THE CLOSET by posting an AI Generated Sequence in the comments of what happens next once we "enter the closet." I'll try to slice them all together for Halloween! Join in below!show more

Yet another scene that would make #AI bros cry…... This one was entirely hand drawn with key frames and main in-betweens done by me. It was my first episode as a director and I wanted to show what could be done with the show in terms of dynamic action. This episode paved the way for all of the #FairlyOddParents movies and established that we COULD, in fact, have big action sequences. It also established me as the “action guy for comedy shows.”show more

Ape Terminal IDO #23 🦍 Sharpe AI - Sharpe... AI Sharpe AI is the 1st web3 project recognized by two multi-trillion dollar firms: Google and Microsoft. Sharpe AI is among the 1st Bittensor ecosystem projects, using the AIT subnet (80x ROI) to mine $TAO. This edge fuels its rapid growth, establishing it as the fastest-growing AI super app. 🟠 Backed by Contango, main investor of Bittensor ($12.7B) 🟠 #1 on Product Hunt globally for AI and Crypto 🟠 100k+ active users With AI emerging as one of the hottest narratives this cycle, Sharpe AI positions itself at the forefront as one of the 1st AI projects with solid backing and an established user base. Sale opens NOW:show more

[LSTM] by Hand ✍️ LSTMs have been the most... effective architecture to process long sequences of data, until our world was taken over by the Transformers. LSTMs belong to the broader family of recurrent neural network (RNNs) that process data sequentially in a recurrent manner. Transformers, on the other hand, abandon recurrence and use self-attention instead to process data concurrently in parallel. Recently, there is renewed interest in recurrence as people realized self-attention doesn’t scale to extremely long sequences, like hundreds of thousands of tokens. Mamba is a good example to bring back recurrence. All of a sudden, it is cool to study LSTMs. How do LSTMs work? [1] Given ↳ 🟨 Input sequence X1, X2, X3 (d = 3) ↳ 🟩 Hidden state h (d = 2) ↳ 🟦 Memory C (d = 2) ↳ Weight matrices Wf, Wc, Wi, Wo Process t = 1 [2] Initialize ↳ Randomly set the previous hidden state h0 to [1, 1] and memory cells C0 to [0.3, -0.5] [3] Linear Transform ↳ Multiply the four weight matrices with the concatenation of current input (X1) and the previous hidden state (h0). ↳ The results are feature values, each is a linear combination of the current input and hidden state. [4] Non-linear Transform ↳ Apply sigmoid σ to obtain gate values (between 0 and 1). • Forget gate (f1): [-4, -6] → [0, 0] • Input gate (i1): [6, 4] → [1, 1] • Output gate (o1): [4, -5] → [1, 0] ↳ Apply tanh to obtain candidate memory values (between -1 and 1) • Candidate memory (C’1): [1, -6] → [0.8, -1] [5] Update Memory ↳ Forget (C0 .* f1): Element-wise multiply the current memory with forget gate values. ↳ Input (C’1 .* o1): Element-wise multiply the “candidate” memory with input gate values. ↳ Update the memory to C1 by adding the two terms above: C0 .* f1 + C’1 .* o1 = C1 [6] Candiate Output ↳ Apply tanh to the new memory C1 to obtain candidate output o’1. [0.8, -1] → [0.7, -0.8] [7] Update Hidden State ↳ Output (o’1 .* o1 → h1): Element-wise multiply the candidate output with the output gate. ↳ The result is updated hidden state h1 ↳ Also, it is the first output. Process t = 2 [8] Initialize ↳ Copy previous hidden state h1 and memory C1 [9] Linear Transform ↳ Repeat [3] [10] Update Memory (C2) ↳ Repeat [4] and [5] [11] Update Hidden State (h2) ↳ Repeat [6] and [7] Process t = 3 [12] Initialize ↳ Copy previous hidden state h2 and memory C2 [13] Linear Transform ↳ Repeat [3] [14] Update Memory (C3) ↳ Repeat [4] and [5] [15] Update Hidden State (h3) ↳ Repeat [6] and [7]show more

E4C: Web3 AI Agent - designed for Esports backed... by Sui has been officially unveiled. This is definitely Next-Gen Esports. We have a series of exciting announcements related to this product that we’ll be sharing with you. Stay tuned. ✍️Comment : "E4C to the moon" 👬 Repost & Tag your 2 friends 🔗 join our Telegram : We will select 5 winners to receive 20U worth of $E4C token. Please leave your SUI wallet address. Johnson Yeh | E4Cshow more

[VAE] by Hand ✍️ A Variational Auto Encoder (VAE)... learns the structure (mean and variance) of hidden features and generates new data from the learned structure. In contrast, GANs only learn to generate new data to fool a discriminator; they may not necessarily know the underlying structure of the data. The International Conference on Learning Representations (ICLR) this year announced its first ever "Test of Time Award" to recognizes the VAE paper, published 10 years ago. This exercise demonstrates how to calculate a VAE by hand. [1] Given: ↳ Three training examples X1, X2, X3 ↳ Copy training examples to the bottom ↳ The purpose is to train the network to reconstruct the training examples. ↳ Since each target is a training example itself, we use the Greek word "auto" which means "self." This crucial step is what makes an autoencoder "auto." [2] Encoder: Layer 1 + ReLU ↳ Multiply inputs with weights and biases ↳ Apply ReLU, crossing out negative values (-1 -> 0) [3] Encoder: Mean and Variance ↳ Multiply features with two sets of weights and biases ↳ 🟩 The first set predicts the means (𝜇) of latent distributions ↳ 🟪 The second set predicts the standard deviation (𝜎) of latent distributions [4] Reparameterization Trick: Random Offset ↳ Sample epsilon ε from the normal distribution with mean = 0 and variance = 1. ↳ The purpose is to randomly pick a offset away from the mean. ↳ Multiply the standard deviation values with epsilon values. ↳ The purpose is to scale the offset by the standard deviation. [5] Reparameterization Trick: Mean + Offset ↳ Add the sampled offset to predicted mean ↳ The result are new parameters or features 🟨 as inputs to the Decoder. [6] Decoder: Layer 1 + ReLU ↳ Multiply input features with weights and biases ↳ Apply ReLU, crossing out negative values. Here, -4 is crossed out. [7] Decoder: Layer 2 ↳ Multiply features with weights and biases ↳ The output is Decoder's attempt to reconstruct the input data X from reparameterized distributions described by 𝜇 and 𝜎. [8]-[10] KL Divergence Loss [8] Loss Gradient: Mean 𝜇 ↳ We want 𝜇 to approach 0. ↳ A lot of math called SGVB simplifies the calculation of loss gradients to simply 𝜇 [9,10] Loss Gradient: Stdev 𝜎 ↳ We want 𝜎 to approach 1. ↳ A lot of math simplifies the calculation to 𝜎 - (1/ 𝜎) [11] Reconstruction Loss ↳ We want the reconstructed data Y (dark 🟧) to be the same as the input data X. ↳ Some math involving Mean Square Error simplifies the calculation to Y - X.show more

Most AI Projects In Crypto Are Liars. Sentient is... different. It is the only project in Web3 pursuing true Artificial General Intelligence. A few facts that set it apart: 1. First-of-its-kind: Sentient is not just building AI tools, but an open, crypto-native path to AGI. 2. Community-owned intelligence: Instead of Big Tech locking up breakthroughs, progress is built openly and shared on-chain. 3. Value capture: Every step forward in intelligence accrues to the ecosystem itself, not to a closed corporation. 4. Product before hype: While dozens of AI projects launch at $300M–$2B FDV with little to show, Sentient is shipping real technology first. 5. Designed to evolve: Its architecture is built to learn, adapt, and scale like a human mind. 6. The singularity play in crypto: Many see Sentient as the Tesla-equivalent for AGI—but crypto-native. In time, people will understand that this isn’t just another AI project. It’s the beginning of decentralized intelligence.show more

mHC from DeepSeek. I implemented it in Excel for... my Frontier AI Seminar. What is mHC? mHC stands for Manifold-Constrained Hyper Connections, published just a few days ago. This paper has quickly become the "first paper to read in 2026" for many in the community. Here's the gist: Residual Network = Neural Network + Skip Connections Hyper Connections = 1 skip connection per block to N weighted skip connections per block Manifold-Constrained = Whatever weight values to weights constrained to sum of 1 I always appreciate the DeepSeek team for releasing their work openly and quickly. That said, this paper introduces significantly more terminology than their earlier papers, and I’m not entirely sure who the intended audience is. Just to name a few: - Residual stream - Sinkhorn–Knopp algorithm - Entropically projected matrices - Birkhoff polytope - Doubly stochastic matrices 🤯 It’s a good reminder that open source is not the same as open knowledge—which is why I’d like to unpack mHC by hand, in Excel, in the next Frontier AI Seminar.show more