正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Wow. This is one of the best interactive sites I’ve seen for learning how LLMs work! 🔥 It starts w/ a clear intro and guides you through every core component: from Embedding, Layer Norm, and Self-Attention to MLPs, Transformer blocks, Softmax, and Output layers. link in 🧵↓

Charly Wargnier

153,980 subscribers

52,092 次观看 • 1 年前 •via X (Twitter)

科学技术新闻政治教育

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

Tech with Mak

82,230 次观看 • 5 个月前

THE BEST visual explainer of how information propagates through a transformer. If you want to have more than intuition about how the Transformer architecture is ruling the LLM world - open-source project explains everything about LLM Transformer Models! - A great resource for anyone looking to gain a deeper understanding of how Transformer-based AI models like GPT work, including: - Self-attention mechanisms - Encoder-decoder architecture - Positional encoding - Multi-head attention

THE BEST visual explainer of how information propagates through a transformer. If you want to have more than intuition about how the Transformer architecture is ruling the LLM world - open-source project explains everything about LLM Transformer Models! - A great resource for anyone looking to gain a deeper understanding of how Transformer-based AI models like GPT work, including: - Self-attention mechanisms - Encoder-decoder architecture - Positional encoding - Multi-head attention

Rohan Paul

106,897 次观看 • 1 年前

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features across positions, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens across feature dimensions, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes across positions, so a feature can see its neighbours. The FFN mixes across feature dimensions, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning

Tom Yeh

25,796 次观看 • 14 天前

Make full use of layer functions! Discover how to use clipping masks, layer masks, and reference layers effectively in your work. This concise guide has all the essential information you need for using layers and coloring effectively. Learn more here↓ #clipstudio

Make full use of layer functions! Discover how to use clipping masks, layer masks, and reference layers effectively in your work. This concise guide has all the essential information you need for using layers and coloring effectively. Learn more here↓ #clipstudio

CLIP STUDIO PAINT

12,493 次观看 • 1 年前

Bit of a BOOKISH NEWSFLASH for you… For the last five years I’ve been researching and writing a book I’ve poured everything I’ve got in to it, and then some It is about this insanely beautiful place… and the people who live and work in it

Bit of a BOOKISH NEWSFLASH for you… For the last five years I’ve been researching and writing a book I’ve poured everything I’ve got in to it, and then some It is about this insanely beautiful place… and the people who live and work in it

James Rebanks

148,486 次观看 • 2 年前

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Yan Chen

33,326 次观看 • 1 年前

- WUTHERING WAVES NEWS - I’ve officially done it! Guides for every single Wuthering Waves character are done! It took a lot of work but excited to jump in and play tomorrow! I hope you guys enjoy them all and thank you for all the support 🥰❤️ #WutheringWaves

- WUTHERING WAVES NEWS - I’ve officially done it! Guides for every single Wuthering Waves character are done! It took a lot of work but excited to jump in and play tomorrow! I hope you guys enjoy them all and thank you for all the support 🥰❤️ #WutheringWaves

SipSipStefen

51,623 次观看 • 2 年前

I can't believe how good this turned out. Recently, I put out my first physics paper proposing that life and the universe might be the result of a law governing how oscillators sync up and couple together. I find it beautiful, awe-inspiring, and deeply meaningful. It aligns with how I have always seen the world. It suggests that there is an arrow of negative entropy, a law governing how life forms in all of its complexity. Learning about it is intimidating. I am self-taught, and have struggled for months to understand and articulate the core concepts and framework. After I put out the paper, I was challenged by Danielle Fong 🔆 to make an 'explorable explanation'. A guide you can walk through, and which will hold your hand through each concept, equation, and principle. It was amazing to make. In fact, I plan to make many more covering the wide range of topics I have been researching and exploring. Both to help me, and because I believe that this is something the world needs right now. Link in comments

I can't believe how good this turned out. Recently, I put out my first physics paper proposing that life and the universe might be the result of a law governing how oscillators sync up and couple together. I find it beautiful, awe-inspiring, and deeply meaningful. It aligns with how I have always seen the world. It suggests that there is an arrow of negative entropy, a law governing how life forms in all of its complexity. Learning about it is intimidating. I am self-taught, and have struggled for months to understand and articulate the core concepts and framework. After I put out the paper, I was challenged by Danielle Fong 🔆 to make an 'explorable explanation'. A guide you can walk through, and which will hold your hand through each concept, equation, and principle. It was amazing to make. In fact, I plan to make many more covering the wide range of topics I have been researching and exploring. Both to help me, and because I believe that this is something the world needs right now. Link in comments

Parzival - ∞/89

23,087 次观看 • 3 个月前

One of the greatest keys to power and consistency in the golf swing is learning how to properly rotate through the ball. Initiating the downswing is one of the most important tools in consistent golf. It is something I have worked on virtually every day of my life, from hitting balls as a young boy in South Africa to competing against the best players in the world. GP #PXGTroops

One of the greatest keys to power and consistency in the golf swing is learning how to properly rotate through the ball. Initiating the downswing is one of the most important tools in consistent golf. It is something I have worked on virtually every day of my life, from hitting balls as a young boy in South Africa to competing against the best players in the world. GP #PXGTroops

GARY PLAYER

51,687 次观看 • 1 个月前

Full Fine-tuning vs. Freezing Layers. Interact 👉 and == Full Fine-tuning == A real network has many — three layers in this example, billions of parameters in a production model. What does fine-tuning look like when you update all of them? That’s full fine-tuning: continue training every weight in the pretrained network on your new task. Every layer’s W gets its own ΔW. Nothing is frozen — every parameter is in play. Think of an MLP as a chain of prerequisites leading to an advanced course. Layer 1 might be Linear Algebra, layer 2 Probability, layer 3 Advanced Machine Learning — each one building on what came before. Fine-tuning is what happens during graduate study: the foundations are already there from undergrad, so you’re not re-learning. Full fine-tuning is reviewing every prerequisite to see what new topics have appeared and what discoveries the field has made since the last time you sat through them. Effective — but exhausting. This diagram shows the same three-layer MLP twice, side by side. On the left, the pretrained network runs on input X: three weight matrices W₁, W₂, W₃, each followed by a ReLU activation. Full fine-tuning gives the model the most freedom to specialize. Every parameter can move — and every parameter that can move must be stored. But not every prerequisite needs revisiting. The further you go back in the chain, the less the material has changed since pretraining — the linear-algebra basics under your computer-vision course are largely the same as they ever were. The next page does exactly that: freeze the prerequisites that haven’t moved, and only refresh the advanced one closest to your specialization. == Freezing Layers == Full fine-tuning reviewed every prerequisite — Linear Algebra, Probability, Advanced ML — to refresh each subject with the latest topics. Effective, but exhausting. Then you realize something. The prerequisites haven’t actually changed that much. Linear Algebra is still Linear Algebra; the matrix decompositions you learned still hold. Probability is still Probability; the distributions and Bayes’ rule haven’t moved. Almost all the new material — the new ideas, the recent discoveries — lives in the advanced layer at the top. That’s freezing layers: keep the prerequisite layers fixed at their pretrained state, and only update the advanced one. In the diagram below, W1 and W2 — the foundational prerequisites — stay frozen. Only W3 — the layer closest to your task-specific output — gets a ΔW.

Full Fine-tuning vs. Freezing Layers. Interact 👉 and == Full Fine-tuning == A real network has many — three layers in this example, billions of parameters in a production model. What does fine-tuning look like when you update all of them? That’s full fine-tuning: continue training every weight in the pretrained network on your new task. Every layer’s W gets its own ΔW. Nothing is frozen — every parameter is in play. Think of an MLP as a chain of prerequisites leading to an advanced course. Layer 1 might be Linear Algebra, layer 2 Probability, layer 3 Advanced Machine Learning — each one building on what came before. Fine-tuning is what happens during graduate study: the foundations are already there from undergrad, so you’re not re-learning. Full fine-tuning is reviewing every prerequisite to see what new topics have appeared and what discoveries the field has made since the last time you sat through them. Effective — but exhausting. This diagram shows the same three-layer MLP twice, side by side. On the left, the pretrained network runs on input X: three weight matrices W₁, W₂, W₃, each followed by a ReLU activation. Full fine-tuning gives the model the most freedom to specialize. Every parameter can move — and every parameter that can move must be stored. But not every prerequisite needs revisiting. The further you go back in the chain, the less the material has changed since pretraining — the linear-algebra basics under your computer-vision course are largely the same as they ever were. The next page does exactly that: freeze the prerequisites that haven’t moved, and only refresh the advanced one closest to your specialization. == Freezing Layers == Full fine-tuning reviewed every prerequisite — Linear Algebra, Probability, Advanced ML — to refresh each subject with the latest topics. Effective, but exhausting. Then you realize something. The prerequisites haven’t actually changed that much. Linear Algebra is still Linear Algebra; the matrix decompositions you learned still hold. Probability is still Probability; the distributions and Bayes’ rule haven’t moved. Almost all the new material — the new ideas, the recent discoveries — lives in the advanced layer at the top. That’s freezing layers: keep the prerequisite layers fixed at their pretrained state, and only update the advanced one. In the diagram below, W1 and W2 — the foundational prerequisites — stay frozen. Only W3 — the layer closest to your task-specific output — gets a ΔW.

Tom Yeh

27,587 次观看 • 3 个月前

Someone sent this in from Elrow. I’ve been to a few of these myself in Ibiza. One in the one of the warehouses along the docks too. Probably the best club night in the world. All the stage starts coming alive and it all starts coming towards you. People on stilts. It’s mad. Crazy to watch when you are twisted. I’d highly recommend it

Someone sent this in from Elrow. I’ve been to a few of these myself in Ibiza. One in the one of the warehouses along the docks too. Probably the best club night in the world. All the stage starts coming alive and it all starts coming towards you. People on stilts. It’s mad. Crazy to watch when you are twisted. I’d highly recommend it

Around Liverpool

32,677 次观看 • 2 个月前

Self Attention vs Cross Attention by hand ✍️ Resize the matrices yourself 👉 Two attention mechanisms, side by side. Both project X into queries; both compute attention via S = Kᵀ × Q and F = V × A. The only difference is the source of K and V. Self attention uses X for everything. Q, K, and V all come from projecting X. Each X token attends to every other X token. The score matrix S is square — 128 × 128. Cross attention uses X for queries and a second sequence E for keys and values. Each X token attends to every E token instead. The score matrix S is rectangular — 64 × 128. Notice what's shared and what's not: X is the same in both — same 36 × 128 input. Q and K share the 16 dimension — that's what makes the dot product Kᵀ × Q valid in either case. V dimensions are independent: self-attention uses 12, cross-attention uses 12. The choice doesn't depend on which mechanism you're using; it depends on what output dimension your downstream layer expects.

Self Attention vs Cross Attention by hand ✍️ Resize the matrices yourself 👉 Two attention mechanisms, side by side. Both project X into queries; both compute attention via S = Kᵀ × Q and F = V × A. The only difference is the source of K and V. Self attention uses X for everything. Q, K, and V all come from projecting X. Each X token attends to every other X token. The score matrix S is square — 128 × 128. Cross attention uses X for queries and a second sequence E for keys and values. Each X token attends to every E token instead. The score matrix S is rectangular — 64 × 128. Notice what's shared and what's not: X is the same in both — same 36 × 128 input. Q and K share the 16 dimension — that's what makes the dot product Kᵀ × Q valid in either case. V dimensions are independent: self-attention uses 12, cross-attention uses 12. The choice doesn't depend on which mechanism you're using; it depends on what output dimension your downstream layer expects.

Tom Yeh

61,300 次观看 • 3 个月前

March 18, 2025 marked the public launch of OptimAI. In one year, it has evolved from a lightweight node layer into a decentralized intelligence infrastructure powering real-time data, compute, and reinforcement for agentic systems. Not just nodes. Not just data. A continuously learning, network-driven intelligence layer. This is infrastructure for a new class of software: autonomous agents that persist, adapt, and operate across environments. Year one established the network. Year two is where it compounds into coordination and value flow. Personal agents. Reinforcement at network scale. Emerging primitives for AgentFi. New layers coming online. 2026 won’t just be about scale, it’s where the network starts to operate. Keep building!

March 18, 2025 marked the public launch of OptimAI. In one year, it has evolved from a lightweight node layer into a decentralized intelligence infrastructure powering real-time data, compute, and reinforcement for agentic systems. Not just nodes. Not just data. A continuously learning, network-driven intelligence layer. This is infrastructure for a new class of software: autonomous agents that persist, adapt, and operate across environments. Year one established the network. Year two is where it compounds into coordination and value flow. Personal agents. Reinforcement at network scale. Emerging primitives for AgentFi. New layers coming online. 2026 won’t just be about scale, it’s where the network starts to operate. Keep building!

OptimAI Network

34,100 次观看 • 4 个月前

yknow a game that has REALLY good camera work tho? METROID DREAD bc the camera in this game is Samus’s best friend, and the choreography is some of the best I’ve ever seen in a platformer highkey this is how Sonic should’ve been moving against the Titans

yknow a game that has REALLY good camera work tho? METROID DREAD bc the camera in this game is Samus’s best friend, and the choreography is some of the best I’ve ever seen in a platformer highkey this is how Sonic should’ve been moving against the Titans

WokeTiger

31,761 次观看 • 1 年前

This has to be one of the best workout videos I’ve seen - and yeah, I’m officially claiming it for the gays 🏳️‍🌈🔥 Who’s coming with me for the next workout routine?👇🏿😁

This has to be one of the best workout videos I’ve seen - and yeah, I’m officially claiming it for the gays 🏳️‍🌈🔥 Who’s coming with me for the next workout routine?👇🏿😁

Sl!m

24,444 次观看 • 2 个月前

gpt-image-2 + Seedance just took AI realism to another level. I've been testing this combo for a few days. The output is genuinely hard to tell apart from real footage. And the best part? I automated the entire workflow. One click. Same quality every time. rt + Comment "workflow" and I'll DM you the link.

gpt-image-2 + Seedance just took AI realism to another level. I've been testing this combo for a few days. The output is genuinely hard to tell apart from real footage. And the best part? I automated the entire workflow. One click. Same quality every time. rt + Comment "workflow" and I'll DM you the link.

Ahmad

43,782 次观看 • 3 个月前

This was us haha. Sreejith PP Arpit Saxena and I built Andrej Karpathy-fy! It takes a paperswithcode link, your current proficiency and gives you a tailored 5 lesson plan to go from where you are to the meaty parts of the paper implementation - like the karpathy video series / labml style. This is a task none of the previous models could do, and O1 was really good at. The best part? The output is an executable ipynb notebook (which you can use minusxai to work through :) ) The below video has apple's recent monocular depth paper. I could actually go through and understand it, with loss functions from the paper and everything!

This was us haha. Sreejith PP Arpit Saxena and I built Andrej Karpathy-fy! It takes a paperswithcode link, your current proficiency and gives you a tailored 5 lesson plan to go from where you are to the meaty parts of the paper implementation - like the karpathy video series / labml style. This is a task none of the previous models could do, and O1 was really good at. The best part? The output is an executable ipynb notebook (which you can use minusxai to work through :) ) The below video has apple's recent monocular depth paper. I could actually go through and understand it, with loss functions from the paper and everything!

Vivek Aithal

200,848 次观看 • 1 年前

In this Korty’s interview with Asake, this part stood out for me. Asake, while prompting his car’s voice assistant to play him one of his songs, first spoke with a slight accent. But the voice assistant couldn’t understand him and he quickly switched to his original, African self. This, coupled with the usage of the Yoruba language in the interview, shows that to cement your place in the world and get the admiration of the entire world, you must be your original self. This is one of the best interviews I’ve seen and I encourage everyone to go see it as well. God bless all the creatives behind this.

In this Korty’s interview with Asake, this part stood out for me. Asake, while prompting his car’s voice assistant to play him one of his songs, first spoke with a slight accent. But the voice assistant couldn’t understand him and he quickly switched to his original, African self. This, coupled with the usage of the Yoruba language in the interview, shows that to cement your place in the world and get the admiration of the entire world, you must be your original self. This is one of the best interviews I’ve seen and I encourage everyone to go see it as well. God bless all the creatives behind this.

Ayo-Elesho

162,519 次观看 • 4 个月前

One of the most satisfying things about a launch vehicle the size of SLS using cryogenic propellants is how much the core stage contracts when fully loaded. Pay attention to the SRB struts and how they've shifted from the time just before LH2 fast-fill and when the core stage reached 100% capacity. SLS is now at flight-level and in replenish mode for both core stage and ICPS LH2 & LOx. The countdown continues. NSF Live Coverage:

One of the most satisfying things about a launch vehicle the size of SLS using cryogenic propellants is how much the core stage contracts when fully loaded. Pay attention to the SRB struts and how they've shifted from the time just before LH2 fast-fill and when the core stage reached 100% capacity. SLS is now at flight-level and in replenish mode for both core stage and ICPS LH2 & LOx. The countdown continues. NSF Live Coverage:

Max Evans

77,210 次观看 • 5 个月前