AlphaFold by hand✍️ Excel ~ I designed this exercise... to show (1) MSA multi-head attention, (2) Pair triangular update, two key components of the EvoFormer architecture.👇Join the AI Math community. Download xlsx.show more

Transformer by hand✍️ Excel ~ I designed this exercise... to show the core math of a Transformer model is to combine columns (attention), combine rows (feed forward), and repeat.👇Join the 'AI Math' community. 👇Download xlsx.show more

LSTM by hand✍️Excel ~ I designed this exercise to... show it is possible to calculate a simple LSTM by hand. Green🟩: Short Term. Blue🟦: Long Term. +equations. +medium.👇Join the 'AI Math' community. Download xlsx.show more

Autoencoder by hand✍️Excel~ I designed this exercise to show... how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. 👇Join the 'AI Math' community. Download xlsx.show more

ResNet by hand✍️Excel~ I designed this exercise to compare... a ResNet to an MLP and show skip-connections are simply identity matrices next to weights and biases. I also made a medium version to show how it scales. 👇Join the 'AI Math' community. Download xlsx.show more

Autoencoder by hand✍️Excel~ I designed this exercise to show... how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. I also made a medium version.👇Join the 'AI Math' community. Download xlsx.show more

Backpropagation by hand✍️ ~ spreadsheet. I designed this exercise... to show it is possible to calculate backpropagation for a non-trivial, three-layer network by hand. p.s. I just started this community to share useful resources on AI math. 👇 Join the community.show more

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~... I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.show more

At MIT, the only course I ever dropped was... signal processing. The DFT math was too intimidating. It’s so easy to just type fft() in MATLAB and move on. Years later, I finally did DFT by hand. ✍️ If you are also afraid of DFT, I hope this helps! ⬇️ Download:show more

At MIT, I learned about RNNs in my NLP... class with Prof. Michael Collins. He built a model from my keystrokes to predict who I was. To me, it felt like a magic box. Years later, when I had to teach RNNs, I forced myself to go inside the box. ⬇️ Download: First, with a tiny example on paper by hand ✍️. Then, a slightly larger one in Excel. That’s when it finally clicked: 👉 The weights are reused (weight matrices on the left side) 👉 The hidden states are passed down (H's) When I built it by hand and saw everything visually, it clicked, just math you can actually trace. Now I try to give others the same “aha!” moment I had. ⬇️ Download Excel:show more

THE BEST visual explainer of how information propagates through... a transformer. If you want to have more than intuition about how the Transformer architecture is ruling the LLM world - open-source project explains everything about LLM Transformer Models! - A great resource for anyone looking to gain a deeper understanding of how Transformer-based AI models like GPT work, including: - Self-attention mechanisms - Encoder-decoder architecture - Positional encoding - Multi-head attentionshow more

▲ @Vercel AI Elements, Voice Edition A brand new... set of components designed to work with the AI SDK to help you build the next generation of voice agents, transcription services and apps powered by natural language. Here's a thread on what's new! 👇show more

IN HAND: adidas Aruku MRBAILEY This pair is limited... to 100 pairs ✍️✍️show more

Good multi-purpose exercise for the neck, shoulders, and back... pain... do this 👇👇👇show more

Single vs Multi-hand Attention by hand ✍️ Resize matrices... yourself 👉 The most important fact about multi-head attention: it has the same parameter count as single-head attention. The difference is purely structural — same total Wqkv weights, partitioned into smaller q–k–v triples. Look at the two diagrams below. Both Wqkv matrices have the same height — same number of weight rows, same number of parameters. What changes is how that single tall block is sliced. • Left. One head. The full Wqkv produces one big QKV: a tall Q (36 rows), a tall K, a tall V. One scoring computation runs over those full-width tensors. • Right. 3 heads. The same-height Wqkv is sliced into 3 smaller q–k–v triples — each 12 rows tall. 3 scoring computations run in parallel, each a thinner version of the left. The compute trade-off — kind of. Same Wqkv weights. Multi-head runs the attention scoring S = Kᵀ × Q once per head, so the dot-product count multiplies by H. • Single-head: seq × seq = 40² = 1600 dot products • Multi-head: seq × seq × H = 40² × 3 = 4800 dot products (3×) But each multi-head dot product is narrower — its inner dimension is head_dim instead of H × head_dim. So when you count actual scalar multiplications, the totals are equal: • Single-head: seq² × (H × head_dim) = 40² × 36 = 57600 • Multi-head: seq² × H × head_dim = 40² × 3 × 12 = 57600 Same FLOPs. Multi-head buys you H independent attention patterns at no extra weight cost and no extra arithmetic cost — it's the same total compute, sliced into H finer-grained heads.show more

🚨 Omg.. the $1M AI War just began. Genspark... launched a $1 MILLION Side-by-Side AI Showdown, challenging any AI, including ChatGPT. I tested many of them on the same tasks… but In every test, Genspark’s results came out on top. Check the head-to-head below: 👇show more

Nique Clifford drives baseline for the two-hand JAM! Kings... trail by 2 as we head into Q4 on Prime... Tap to watch:show more

HOLY CRAP! I can't tell you how big this... is for the medical community and drug discovery: Google Announces AlphaFold 3 AI. Details: Enhanced Molecular Prediction: AlphaFold 3 predicts the structure and interactions of all life's molecules, including proteins, DNA, RNA, ligands, and more, with unprecedented accuracy. Improved Interaction Accuracy: For protein interactions with other molecule types, AlphaFold 3 offers at least a 50% improvement over existing methods, and doubles the accuracy for some critical interactions. Transformative Potential for Science and Medicine: The model aims to deepen our understanding of biological processes and significantly advance drug discovery efforts. Accessibility for Researchers: AlphaFold 3's capabilities are largely accessible for free via the AlphaFold Server, providing an essential tool for scientific research. Drug Design Innovation: AlphaFold 3 is utilized by Isomorphic Labs in collaboration with pharmaceutical companies to accelerate drug design, potentially leading to new treatments for various diseases. Foundation in AlphaFold 2: Building on the breakthroughs of AlphaFold 2, this version extends its scope beyond proteins to a wide range of biomolecules, enhancing its utility in scientific research and application. Global Accessibility and Educational Support: The AlphaFold Server is a free platform for non-commercial research worldwide, supported by educational resources to foster wider adoption and innovation. Empowering Rapid Scientific Advancements: By making detailed molecular interactions easily accessible, AlphaFold 3 enables faster hypothesis testing and could reduce the time and cost typically associated with experimental protein-structure prediction. Responsible Development and Deployment: DeepMind has engaged with domain experts to assess the impacts and potential risks of AlphaFold, ensuring its responsible use in the scientific community. Broad Implications for Biology:AlphaFold 3 helps reveal complex cellular mechanisms and interactions, offering insights that could lead to improved agricultural crops, enhanced understanding of diseases, and novel therapeutic strategies.show more

The “IPs” of the future will be community-built, brought... to life by AI, and attention-optimized by the market. We must make own worlds or be forced to live in someone else’s.show more

Here are two new Unique items coming to Path... of Exile 2 in the 0.2.0 update! Embrace your Gladiatorial fighting spirit by using The Bringer of Rain or show your devotion to the Gods of Wraeclast, old or new, with the Hammer of Faith Two-Handed Mace.show more