Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

I spent months illustrating how Transformers actually work. Not just what they do, but why they’re built this way. The history, design choices, and intuition behind every layer. From RNNs → Attention → Multi-Head → FFNs → Positional Encoding. Here's everything I wish I knew:

krupa

2,994 subscribers

250,477 Aufrufe • vor 9 Monaten •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Vindicated! (lol) I knew I was not misremembering things with Star Fox from back in the day. This is kinda like what I was saying on the Spawncast. The movie design is cute, but the realstic design is what Star Fox actually was and should be imo.

Vindicated! (lol) I knew I was not misremembering things with Star Fox from back in the day. This is kinda like what I was saying on the Spawncast. The movie design is cute, but the realstic design is what Star Fox actually was and should be imo.

OJ - PlayerEssence

32,003 Aufrufe • vor 2 Monaten

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,220 Aufrufe • vor 1 Jahr

I wish I knew how they did this, because it's a total mystery to me. There's no way I can figure out how they seem to plan and know what the end product will be. I guess some people are blessed with that kind of mind & imagination... But I am sure not one of them.

I wish I knew how they did this, because it's a total mystery to me. There's no way I can figure out how they seem to plan and know what the end product will be. I guess some people are blessed with that kind of mind & imagination... But I am sure not one of them.

J∅kër Kîng 👑

47,877 Aufrufe • vor 6 Monaten

“Governor Alex Otti is just repainting and rebuilding the roads I built. What he has achieved in four months is what I achieved in eight years. I have not seen any new work from him. Because I said I will work for APC, that is why I am receiving insvlts, and it is not acceptable to me.” —Senator Orji Uzor Kalu

“Governor Alex Otti is just repainting and rebuilding the roads I built. What he has achieved in four months is what I achieved in eight years. I have not seen any new work from him. Because I said I will work for APC, that is why I am receiving insvlts, and it is not acceptable to me.” —Senator Orji Uzor Kalu

CHUKS 🍥

448,862 Aufrufe • vor 6 Monaten

last weekend at NS, i broke down everything i've learned about writing bangers into a single session → how hooks actually work → why most tweets die in 0.3 seconds → the storytelling patterns behind every viral post and the one framework that changed how i think about content forever. here's the full playbook:

last weekend at NS, i broke down everything i've learned about writing bangers into a single session → how hooks actually work → why most tweets die in 0.3 seconds → the storytelling patterns behind every viral post and the one framework that changed how i think about content forever. here's the full playbook:

Priyanshu Ratnakar

10,550 Aufrufe • vor 5 Monaten

LLMs can make sense of retrieved context because of how transformers work. In one of the lessons from the Retrieval Augmented Generation (RAG) course, we unpack how LLMs process augmented prompts using token embeddings, positional vectors, and multi-head attention. Understanding these internals helps you design more reliable and efficient RAG systems. Watch the breakdown and keep learning how to build production-ready RAG systems in this course, taught by Zain:

LLMs can make sense of retrieved context because of how transformers work. In one of the lessons from the Retrieval Augmented Generation (RAG) course, we unpack how LLMs process augmented prompts using token embeddings, positional vectors, and multi-head attention. Understanding these internals helps you design more reliable and efficient RAG systems. Watch the breakdown and keep learning how to build production-ready RAG systems in this course, taught by Zain:

DeepLearning.AI

11,500 Aufrufe • vor 11 Monaten

Just remember what fear they put into people with Covid. I remember this video distinctly and I knew when I saw the lady with shopping bags on her head, that it was not a real thing. Wonder what these sprays actually did to the kids.

Just remember what fear they put into people with Covid. I remember this video distinctly and I knew when I saw the lady with shopping bags on her head, that it was not a real thing. Wonder what these sprays actually did to the kids.

Enzo The Baker

28,830 Aufrufe • vor 2 Jahren

— [ PLEASE WATCH ] — I know I'm not on this site anymore, but here's a super quick video of what I do, where to find my work, and how to support me! ^^

— [ PLEASE WATCH ] — I know I'm not on this site anymore, but here's a super quick video of what I do, where to find my work, and how to support me! ^^

VBee 🎙️ MOVED TO BSKY

16,447 Aufrufe • vor 2 Jahren

building large language models from scratch by Sebastian Raschka was a great chance for me to sit down and study again all the LLM basics > token and positional embeddings > self-attention and what QKV is about > causal & multi-head attention studying llms and how they work can seem overwhelming at first. but once you taste how good it feels to learn these things intuitively there's no going back. I shared the resources and my notes on my repo and I hope it's a motivation if you want to start as well or recap.

building large language models from scratch by Sebastian Raschka was a great chance for me to sit down and study again all the LLM basics > token and positional embeddings > self-attention and what QKV is about > causal & multi-head attention studying llms and how they work can seem overwhelming at first. but once you taste how good it feels to learn these things intuitively there's no going back. I shared the resources and my notes on my repo and I hope it's a motivation if you want to start as well or recap.

ℏεsam

49,292 Aufrufe • vor 1 Jahr

rosé: I’m not joking but there were so many people online who would tell me what to do with the way i put my makeup on the way I was dressed in stage they would like harass my hair and makeup team online. those people are people i love and that work with me and that’s part of my team and those are my best friends too. but I’ve felt the ache where when people on online I feel like they know what they’re doing they’re really mean to them and that like affects me and they be like do this to your hair, do that to your hair, why are you not doing this and that.

rosé: I’m not joking but there were so many people online who would tell me what to do with the way i put my makeup on the way I was dressed in stage they would like harass my hair and makeup team online. those people are people i love and that work with me and that’s part of my team and those are my best friends too. but I’ve felt the ache where when people on online I feel like they know what they’re doing they’re really mean to them and that like affects me and they be like do this to your hair, do that to your hair, why are you not doing this and that.

rosé pics

45,748 Aufrufe • vor 5 Monaten

🎙️Why did you choose this program? 🐹 Honestly, after I was discharged from the military, one of the things I worried about a lot was how I’d heard from friends who were discharged earlier that “there’s no work for a while.” 🐹 But oh wow, what do you know! 🐹 They reached out to me right away, so I didn’t even need a reason. 🐹 I just thought, “I have to do this no matter what.” 🐹 And since I like cooking, I did have some worries, but I went into it thinking, “Let’s just face it head-on.” #KIHYUN #기현

🎙️Why did you choose this program? 🐹 Honestly, after I was discharged from the military, one of the things I worried about a lot was how I’d heard from friends who were discharged earlier that “there’s no work for a while.” 🐹 But oh wow, what do you know! 🐹 They reached out to me right away, so I didn’t even need a reason. 🐹 I just thought, “I have to do this no matter what.” 🐹 And since I like cooking, I did have some worries, but I went into it thinking, “Let’s just face it head-on.” #KIHYUN #기현

💜IM with Yoo💙

12,355 Aufrufe • vor 1 Jahr

Steve Kerr on LeBron - “I was blown away at how hard he practices” “And I said Spo what about way back when you got him? He said everyday, every drill, every walk through” “It’s not just the talent, it’s the incredible work ethic and attention to detail”

Steve Kerr on LeBron - “I was blown away at how hard he practices” “And I said Spo what about way back when you got him? He said everyday, every drill, every walk through” “It’s not just the talent, it’s the incredible work ethic and attention to detail”

Ryan Pannone

226,196 Aufrufe • vor 1 Jahr

The old Parliament was built by the British but the brain and work behind the new Parliament is of Indians and I don’t understand, why the opposition is making so many issues. They’re upset with the fact PM Modi has done what they (opposition) couldn’t. #NewParliamentBuilding

The old Parliament was built by the British but the brain and work behind the new Parliament is of Indians and I don’t understand, why the opposition is making so many issues. They’re upset with the fact PM Modi has done what they (opposition) couldn’t. #NewParliamentBuilding

Navnit Ravi Rana

11,662 Aufrufe • vor 3 Jahren

“Governor Alex Otti Is Just Repainting And Rebuilding The Roads I Built. What He Has Achieved In Four Months Is What I Achieved In Eight Years. I Have Not Seen Any New Work From Him. Because I Said I Would Work For APC, I Am Receiving Insults, And That Is Unacceptable To Me”. ~ Senator Orji Uzor Kalu.

“Governor Alex Otti Is Just Repainting And Rebuilding The Roads I Built. What He Has Achieved In Four Months Is What I Achieved In Eight Years. I Have Not Seen Any New Work From Him. Because I Said I Would Work For APC, I Am Receiving Insults, And That Is Unacceptable To Me”. ~ Senator Orji Uzor Kalu.

Somto Okonkwo

67,697 Aufrufe • vor 6 Monaten

Hype keeps building for Kentucky. Big Blue Nation is a major reason why. This fan base isn’t really understood outside of the local bubble. They want football success — badly. “I thought I knew, but I did not know. They’re psychotic in the best way.”

Hype keeps building for Kentucky. Big Blue Nation is a major reason why. This fan base isn’t really understood outside of the local bubble. They want football success — badly. “I thought I knew, but I did not know. They’re psychotic in the best way.”

Adam Luckett

88,860 Aufrufe • vor 3 Monaten

#LUNAR #filmracha ❤️‍🩹: Film… now completely understands how a mother feels. I’m not sure if I can put it this way, but after spent a whole day watching Lunar work, I'm just so, so proud of her. I understand everything now, I completely get how the fans feel. You're just so adorable, you know that, Lunar?

#LUNAR #filmracha ❤️‍🩹: Film… now completely understands how a mother feels. I’m not sure if I can put it this way, but after spent a whole day watching Lunar work, I'm just so, so proud of her. I understand everything now, I completely get how the fans feel. You're just so adorable, you know that, Lunar?

แมวแมวเอง

42,351 Aufrufe • vor 1 Monat

Q from Frank Thorp V: Why do you think the Trump admin is so fixated on MN right now? TINA SMITH: "Well I wish I knew. I mean I wish they would just leave us the f*** alone. I mean, seriously, when do things stop being about politics and start being about actual human decency"

Q from Frank Thorp V: Why do you think the Trump admin is so fixated on MN right now? TINA SMITH: "Well I wish I knew. I mean I wish they would just leave us the f*** alone. I mean, seriously, when do things stop being about politics and start being about actual human decency"

Alan He

159,978 Aufrufe • vor 6 Monaten

The way they love their members already says everything but they’re just as warm with their families, staff, fans, and everyone they work with… it’s one of the main reasons I fell for them and stayed 🥹

The way they love their members already says everything but they’re just as warm with their families, staff, fans, and everyone they work with… it’s one of the main reasons I fell for them and stayed 🥹

hohocham

132,375 Aufrufe • vor 4 Monaten

💬 what do you feel when they hated on you because you existed 🌙 i don't know, honestly there's not much that I could say but you guys really really really, not you guys not my fans but you know, just people in general really like talking about the way I look or what I lack in. I do see that shit of course I do but it's uhm.. it's kinda.. how do i say this without sounding.. bad. It's kinda sad you know, i wish a lot more people have a lot more love in their hearts, but yeah

💬 what do you feel when they hated on you because you existed 🌙 i don't know, honestly there's not much that I could say but you guys really really really, not you guys not my fans but you know, just people in general really like talking about the way I look or what I lack in. I do see that shit of course I do but it's uhm.. it's kinda.. how do i say this without sounding.. bad. It's kinda sad you know, i wish a lot more people have a lot more love in their hearts, but yeah

•ᴗ•

101,785 Aufrufe • vor 1 Monat