Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

AlphaFold by hand✍️ Excel ~ I designed this exercise to show (1) MSA multi-head attention, (2) Pair triangular update, two key components of the EvoFormer architecture.👇Join the AI Math community. Download xlsx.

Tom Yeh

56,051 subscribers

104,965 görüntüleme • 1 yıl önce •via X (Twitter)

Eğitim Sağlık & İyilik Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

LSTM by hand✍️Excel ~ I designed this exercise to show it is possible to calculate a simple LSTM by hand. Green🟩: Short Term. Blue🟦: Long Term. +equations. +medium.👇Join the 'AI Math' community. Download xlsx.

LSTM by hand✍️Excel ~ I designed this exercise to show it is possible to calculate a simple LSTM by hand. Green🟩: Short Term. Blue🟦: Long Term. +equations. +medium.👇Join the 'AI Math' community. Download xlsx.

Tom Yeh

41,586 görüntüleme • 1 yıl önce

Autoencoder by hand✍️Excel~ I designed this exercise to show how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. 👇Join the 'AI Math' community. Download xlsx.

Autoencoder by hand✍️Excel~ I designed this exercise to show how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. 👇Join the 'AI Math' community. Download xlsx.

Tom Yeh

101,555 görüntüleme • 1 yıl önce

ResNet by hand✍️Excel~ I designed this exercise to compare a ResNet to an MLP and show skip-connections are simply identity matrices next to weights and biases. I also made a medium version to show how it scales. 👇Join the 'AI Math' community. Download xlsx.

ResNet by hand✍️Excel~ I designed this exercise to compare a ResNet to an MLP and show skip-connections are simply identity matrices next to weights and biases. I also made a medium version to show how it scales. 👇Join the 'AI Math' community. Download xlsx.

Tom Yeh

29,913 görüntüleme • 1 yıl önce

Autoencoder by hand✍️Excel~ I designed this exercise to show how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. I also made a medium version.👇Join the 'AI Math' community. Download xlsx.

Autoencoder by hand✍️Excel~ I designed this exercise to show how an Encoder-Decoder network convert input to code and reconstruct input from code. It is annotated with equations, PyTorch, and graphs. I also made a medium version.👇Join the 'AI Math' community. Download xlsx.

Tom Yeh

54,473 görüntüleme • 1 yıl önce

Backpropagation by hand✍️ ~ spreadsheet. I designed this exercise to show it is possible to calculate backpropagation for a non-trivial, three-layer network by hand. p.s. I just started this community to share useful resources on AI math. 👇 Join the community.

Backpropagation by hand✍️ ~ spreadsheet. I designed this exercise to show it is possible to calculate backpropagation for a non-trivial, three-layer network by hand. p.s. I just started this community to share useful resources on AI math. 👇 Join the community.

Tom Yeh

121,825 görüntüleme • 1 yıl önce

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Yan Chen

33,255 görüntüleme • 1 yıl önce

At MIT, the only course I ever dropped was signal processing. The DFT math was too intimidating. It’s so easy to just type fft() in MATLAB and move on. Years later, I finally did DFT by hand. ✍️ If you are also afraid of DFT, I hope this helps! ⬇️ Download:

At MIT, the only course I ever dropped was signal processing. The DFT math was too intimidating. It’s so easy to just type fft() in MATLAB and move on. Years later, I finally did DFT by hand. ✍️ If you are also afraid of DFT, I hope this helps! ⬇️ Download:

Tom Yeh

260,027 görüntüleme • 8 ay önce

At MIT, I learned about RNNs in my NLP class with Prof. Michael Collins. He built a model from my keystrokes to predict who I was. To me, it felt like a magic box. Years later, when I had to teach RNNs, I forced myself to go inside the box. ⬇️ Download: First, with a tiny example on paper by hand ✍️. Then, a slightly larger one in Excel. That’s when it finally clicked: 👉 The weights are reused (weight matrices on the left side) 👉 The hidden states are passed down (H's) When I built it by hand and saw everything visually, it clicked, just math you can actually trace. Now I try to give others the same “aha!” moment I had. ⬇️ Download Excel:

At MIT, I learned about RNNs in my NLP class with Prof. Michael Collins. He built a model from my keystrokes to predict who I was. To me, it felt like a magic box. Years later, when I had to teach RNNs, I forced myself to go inside the box. ⬇️ Download: First, with a tiny example on paper by hand ✍️. Then, a slightly larger one in Excel. That’s when it finally clicked: 👉 The weights are reused (weight matrices on the left side) 👉 The hidden states are passed down (H's) When I built it by hand and saw everything visually, it clicked, just math you can actually trace. Now I try to give others the same “aha!” moment I had. ⬇️ Download Excel:

Tom Yeh

220,627 görüntüleme • 8 ay önce

THE BEST visual explainer of how information propagates through a transformer. If you want to have more than intuition about how the Transformer architecture is ruling the LLM world - open-source project explains everything about LLM Transformer Models! - A great resource for anyone looking to gain a deeper understanding of how Transformer-based AI models like GPT work, including: - Self-attention mechanisms - Encoder-decoder architecture - Positional encoding - Multi-head attention

THE BEST visual explainer of how information propagates through a transformer. If you want to have more than intuition about how the Transformer architecture is ruling the LLM world - open-source project explains everything about LLM Transformer Models! - A great resource for anyone looking to gain a deeper understanding of how Transformer-based AI models like GPT work, including: - Self-attention mechanisms - Encoder-decoder architecture - Positional encoding - Multi-head attention

Rohan Paul

106,880 görüntüleme • 1 yıl önce

▲ @Vercel AI Elements, Voice Edition A brand new set of components designed to work with the AI SDK to help you build the next generation of voice agents, transcription services and apps powered by natural language. Here's a thread on what's new! 👇

Hayden Bleasel

90,486 görüntüleme • 5 ay önce

IN HAND: adidas Aruku MRBAILEY This pair is limited to 100 pairs ✍️✍️

IN HAND: adidas Aruku MRBAILEY This pair is limited to 100 pairs ✍️✍️

Sole Retriever

12,826 görüntüleme • 7 ay önce

Good multi-purpose exercise for the neck, shoulders, and back pain... do this 👇👇👇

Good multi-purpose exercise for the neck, shoulders, and back pain... do this 👇👇👇

Life Doctor

380,118 görüntüleme • 1 yıl önce

Single vs Multi-hand Attention by hand ✍️ Resize matrices yourself 👉 The most important fact about multi-head attention: it has the same parameter count as single-head attention. The difference is purely structural — same total Wqkv weights, partitioned into smaller q–k–v triples. Look at the two diagrams below. Both Wqkv matrices have the same height — same number of weight rows, same number of parameters. What changes is how that single tall block is sliced. • Left. One head. The full Wqkv produces one big QKV: a tall Q (36 rows), a tall K, a tall V. One scoring computation runs over those full-width tensors. • Right. 3 heads. The same-height Wqkv is sliced into 3 smaller q–k–v triples — each 12 rows tall. 3 scoring computations run in parallel, each a thinner version of the left. The compute trade-off — kind of. Same Wqkv weights. Multi-head runs the attention scoring S = Kᵀ × Q once per head, so the dot-product count multiplies by H. • Single-head: seq × seq = 40² = 1600 dot products • Multi-head: seq × seq × H = 40² × 3 = 4800 dot products (3×) But each multi-head dot product is narrower — its inner dimension is head_dim instead of H × head_dim. So when you count actual scalar multiplications, the totals are equal: • Single-head: seq² × (H × head_dim) = 40² × 36 = 57600 • Multi-head: seq² × H × head_dim = 40² × 3 × 12 = 57600 Same FLOPs. Multi-head buys you H independent attention patterns at no extra weight cost and no extra arithmetic cost — it's the same total compute, sliced into H finer-grained heads.

Single vs Multi-hand Attention by hand ✍️ Resize matrices yourself 👉 The most important fact about multi-head attention: it has the same parameter count as single-head attention. The difference is purely structural — same total Wqkv weights, partitioned into smaller q–k–v triples. Look at the two diagrams below. Both Wqkv matrices have the same height — same number of weight rows, same number of parameters. What changes is how that single tall block is sliced. • Left. One head. The full Wqkv produces one big QKV: a tall Q (36 rows), a tall K, a tall V. One scoring computation runs over those full-width tensors. • Right. 3 heads. The same-height Wqkv is sliced into 3 smaller q–k–v triples — each 12 rows tall. 3 scoring computations run in parallel, each a thinner version of the left. The compute trade-off — kind of. Same Wqkv weights. Multi-head runs the attention scoring S = Kᵀ × Q once per head, so the dot-product count multiplies by H. • Single-head: seq × seq = 40² = 1600 dot products • Multi-head: seq × seq × H = 40² × 3 = 4800 dot products (3×) But each multi-head dot product is narrower — its inner dimension is head_dim instead of H × head_dim. So when you count actual scalar multiplications, the totals are equal: • Single-head: seq² × (H × head_dim) = 40² × 36 = 57600 • Multi-head: seq² × H × head_dim = 40² × 3 × 12 = 57600 Same FLOPs. Multi-head buys you H independent attention patterns at no extra weight cost and no extra arithmetic cost — it's the same total compute, sliced into H finer-grained heads.

Tom Yeh

35,116 görüntüleme • 1 ay önce

🚨 Omg.. the $1M AI War just began. Genspark launched a $1 MILLION Side-by-Side AI Showdown, challenging any AI, including ChatGPT. I tested many of them on the same tasks… but In every test, Genspark’s results came out on top. Check the head-to-head below: 👇

🚨 Omg.. the $1M AI War just began. Genspark launched a $1 MILLION Side-by-Side AI Showdown, challenging any AI, including ChatGPT. I tested many of them on the same tasks… but In every test, Genspark’s results came out on top. Check the head-to-head below: 👇

ZOYA ✪

105,301 görüntüleme • 10 ay önce

Nique Clifford drives baseline for the two-hand JAM! Kings trail by 2 as we head into Q4 on Prime... Tap to watch:

Nique Clifford drives baseline for the two-hand JAM! Kings trail by 2 as we head into Q4 on Prime... Tap to watch:

NBA

65,208 görüntüleme • 4 ay önce

HOLY CRAP! I can't tell you how big this is for the medical community and drug discovery: Google Announces AlphaFold 3 AI. Details: Enhanced Molecular Prediction: AlphaFold 3 predicts the structure and interactions of all life's molecules, including proteins, DNA, RNA, ligands, and more, with unprecedented accuracy. Improved Interaction Accuracy: For protein interactions with other molecule types, AlphaFold 3 offers at least a 50% improvement over existing methods, and doubles the accuracy for some critical interactions. Transformative Potential for Science and Medicine: The model aims to deepen our understanding of biological processes and significantly advance drug discovery efforts. Accessibility for Researchers: AlphaFold 3's capabilities are largely accessible for free via the AlphaFold Server, providing an essential tool for scientific research. Drug Design Innovation: AlphaFold 3 is utilized by Isomorphic Labs in collaboration with pharmaceutical companies to accelerate drug design, potentially leading to new treatments for various diseases. Foundation in AlphaFold 2: Building on the breakthroughs of AlphaFold 2, this version extends its scope beyond proteins to a wide range of biomolecules, enhancing its utility in scientific research and application. Global Accessibility and Educational Support: The AlphaFold Server is a free platform for non-commercial research worldwide, supported by educational resources to foster wider adoption and innovation. Empowering Rapid Scientific Advancements: By making detailed molecular interactions easily accessible, AlphaFold 3 enables faster hypothesis testing and could reduce the time and cost typically associated with experimental protein-structure prediction. Responsible Development and Deployment: DeepMind has engaged with domain experts to assess the impacts and potential risks of AlphaFold, ensuring its responsible use in the scientific community. Broad Implications for Biology:AlphaFold 3 helps reveal complex cellular mechanisms and interactions, offering insights that could lead to improved agricultural crops, enhanced understanding of diseases, and novel therapeutic strategies.

HOLY CRAP! I can't tell you how big this is for the medical community and drug discovery: Google Announces AlphaFold 3 AI. Details: Enhanced Molecular Prediction: AlphaFold 3 predicts the structure and interactions of all life's molecules, including proteins, DNA, RNA, ligands, and more, with unprecedented accuracy. Improved Interaction Accuracy: For protein interactions with other molecule types, AlphaFold 3 offers at least a 50% improvement over existing methods, and doubles the accuracy for some critical interactions. Transformative Potential for Science and Medicine: The model aims to deepen our understanding of biological processes and significantly advance drug discovery efforts. Accessibility for Researchers: AlphaFold 3's capabilities are largely accessible for free via the AlphaFold Server, providing an essential tool for scientific research. Drug Design Innovation: AlphaFold 3 is utilized by Isomorphic Labs in collaboration with pharmaceutical companies to accelerate drug design, potentially leading to new treatments for various diseases. Foundation in AlphaFold 2: Building on the breakthroughs of AlphaFold 2, this version extends its scope beyond proteins to a wide range of biomolecules, enhancing its utility in scientific research and application. Global Accessibility and Educational Support: The AlphaFold Server is a free platform for non-commercial research worldwide, supported by educational resources to foster wider adoption and innovation. Empowering Rapid Scientific Advancements: By making detailed molecular interactions easily accessible, AlphaFold 3 enables faster hypothesis testing and could reduce the time and cost typically associated with experimental protein-structure prediction. Responsible Development and Deployment: DeepMind has engaged with domain experts to assess the impacts and potential risks of AlphaFold, ensuring its responsible use in the scientific community. Broad Implications for Biology:AlphaFold 3 helps reveal complex cellular mechanisms and interactions, offering insights that could lead to improved agricultural crops, enhanced understanding of diseases, and novel therapeutic strategies.

Brian Krassenstein

258,536 görüntüleme • 2 yıl önce

The “IPs” of the future will be community-built, brought to life by AI, and attention-optimized by the market. We must make own worlds or be forced to live in someone else’s.

The “IPs” of the future will be community-built, brought to life by AI, and attention-optimized by the market. We must make own worlds or be forced to live in someone else’s.

DreamNet

74,630 görüntüleme • 1 yıl önce

Here are two new Unique items coming to Path of Exile 2 in the 0.2.0 update! Embrace your Gladiatorial fighting spirit by using The Bringer of Rain or show your devotion to the Gods of Wraeclast, old or new, with the Hammer of Faith Two-Handed Mace.

Here are two new Unique items coming to Path of Exile 2 in the 0.2.0 update! Embrace your Gladiatorial fighting spirit by using The Bringer of Rain or show your devotion to the Gods of Wraeclast, old or new, with the Hammer of Faith Two-Handed Mace.

Path of Exile

285,575 görüntüleme • 1 yıl önce

Blades of the Guardians episode two is crazy. There are many battles, but I would like to draw attention to this one. We also have hand to hand combat, and it's just as good. Strong choreography with great use of the 3D camera, and the variety of techniques is impressive.

Blades of the Guardians episode two is crazy. There are many battles, but I would like to draw attention to this one. We also have hand to hand combat, and it's just as good. Strong choreography with great use of the 3D camera, and the variety of techniques is impressive.

Oleksandr

13,820 görüntüleme • 1 yıl önce