Загрузка видео...

Не удалось загрузить видео

На главную

Self-Attention vs Cross-Attention interactive diagram. Open

ProfTomYeh's profile picture

Tom Yeh

65,344 subscribers

20,878 просмотров • 3 дней назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Last week, we launched "Attention in Transformers: Concepts and Code in PyTorch" instructed by Joshua Starmer! In this course, you'll: ✅ Learn how the attention mechanism in LLMs helps convert base token embeddings into rich context-aware embeddings. ✅ Understand the Query, Key, and Value matrices, what they are for, how to produce them, and how to use them in attention. ✅ Learn the difference between self-attention, masked self-attention, and cross-attention, and how multi-head attention scales the algorithm. 🔗 Enroll for free:
1:09

Last week, we launched "Attention in Transformers: Concepts and Code in PyTorch" instructed by Joshua Starmer! In this course, you'll: ✅ Learn how the attention mechanism in LLMs helps convert base token embeddings into rich context-aware embeddings. ✅ Understand the Query, Key, and Value matrices, what they are for, how to produce them, and how to use them in attention. ✅ Learn the difference between self-attention, masked self-attention, and cross-attention, and how multi-head attention scales the algorithm. 🔗 Enroll for free:

DeepLearning.AI

36,832 просмотров • 1 год назад

Seedance 2.0 is impressive. But it's closed-source! Introducing our daVinci-MagiHuman — a single-stream 15B Transformer trained from scratch that jointly generates video + audio. No cross-attention. No multi-stream branches. Just self-attention. ⚡ 5s 1080p video in 38s on a single H100 🏆 80% win rate vs Ovi 1.1 | 60.9% vs LTX 2.3 (2,000 human comparisons) 🌍 6 languages 📦 Fully open-source Speed by simplicity. By @SII_GAIR × Sand.ai 📄 💻 🤗
2:03

Seedance 2.0 is impressive. But it's closed-source! Introducing our daVinci-MagiHuman — a single-stream 15B Transformer trained from scratch that jointly generates video + audio. No cross-attention. No multi-stream branches. Just self-attention. ⚡ 5s 1080p video in 38s on a single H100 🏆 80% win rate vs Ovi 1.1 | 60.9% vs LTX 2.3 (2,000 human comparisons) 🌍 6 languages 📦 Fully open-source Speed by simplicity. By @SII_GAIR × Sand.ai 📄 💻 🤗

Pengfei Liu

300,272 просмотров • 5 месяцев назад

self-attention explained by 3blue1brown's new video
1:51

self-attention explained by 3blue1brown's new video

ℏεsam

48,349 просмотров • 1 год назад

🚨 ATTENTION 🚨 GAVIN CROSS HAS NOW HOMERED IN FIVE STRAIGHT GAMES!
0:22

🚨 ATTENTION 🚨 GAVIN CROSS HAS NOW HOMERED IN FIVE STRAIGHT GAMES!

Omaha Storm Chasers

83,236 просмотров • 1 месяц назад

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:
2:41

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,400 просмотров • 1 год назад

Motions that grab attention. 📩 open for work!
0:21

Motions that grab attention. 📩 open for work!

ARIA

12,439 просмотров • 8 месяцев назад

Motions that grab attention. 📩 open for work!
0:25

Motions that grab attention. 📩 open for work!

ARIA

16,382 просмотров • 8 месяцев назад

Pay attention to how Andrew Tate responds to this question… SELF HYPNOSIS…⚡️
0:40

Pay attention to how Andrew Tate responds to this question… SELF HYPNOSIS…⚡️

Tyson

41,673 просмотров • 9 месяцев назад

Pay deeper attention to the people around you, stop being so self absorbed.
0:56

Pay deeper attention to the people around you, stop being so self absorbed.

Robert Greene

223,238 просмотров • 3 лет назад

Pretending to self-harm yourself in prison for attention. The lowest is what?
0:37

Sensitive content

Pretending to self-harm yourself in prison for attention. The lowest is what?

Boochie is the Name

94,205 просмотров • 1 год назад

Nick Aldis (c) vs Cody Rhodes NWA World Heavyweight Championship All In 2018 🎶 Attention Attention - Shinedown 🎶 Cody Rhodes Nick Aldis
3:06

Nick Aldis (c) vs Cody Rhodes NWA World Heavyweight Championship All In 2018 🎶 Attention Attention - Shinedown 🎶 Cody Rhodes Nick Aldis

CV Highlights

104,958 просмотров • 1 месяц назад

Pay attention when people are offended by the cross. It’s really all you need to know.
2:13

Pay attention when people are offended by the cross. It’s really all you need to know.

TheLizVariant

29,775 просмотров • 7 месяцев назад

A lady intentionally exposed herself while dancing. Attention-seeking or self-expression? What's your take?
1:21

A lady intentionally exposed herself while dancing. Attention-seeking or self-expression? What's your take?

Ella Nnabugo

16,459 просмотров • 1 месяц назад

Easily create image maps on WordPress with the Draw Attention plugin. Immerse your audience with interactive content! Visit the demo below.
0:29

Easily create image maps on WordPress with the Draw Attention plugin. Immerse your audience with interactive content! Visit the demo below.

WP Draw Attention

8,384,147 просмотров • 11 месяцев назад

Drake vs UMG is bigger then Kendrick vs Drake . Yall dot fans relax & pay attention
1:27

Drake vs UMG is bigger then Kendrick vs Drake . Yall dot fans relax & pay attention

lifestyle

20,324 просмотров • 10 месяцев назад

Attention Innovators💡 Applications for National Startup Awards 2023 are open. 👉
0:32

Attention Innovators💡 Applications for National Startup Awards 2023 are open. 👉

Piyush Goyal

586,545 просмотров • 3 лет назад

Turn any GitHub repository into a visual treat! Simply replace "hub" with "diagram" in a GitHub URL and instantly view the entire codebase as interactive diagrams for easier understanding. 100% open-source.
0:21

Turn any GitHub repository into a visual treat! Simply replace "hub" with "diagram" in a GitHub URL and instantly view the entire codebase as interactive diagrams for easier understanding. 100% open-source.

Akshay 🚀

125,479 просмотров • 1 год назад

Attention Attention
0:40

Attention Attention

Zzizinga Dickson ™

253,482 просмотров • 3 лет назад

Attention! Attention!
1:24

Sensitive content

Attention! Attention!

Marceline Cameron

1,070,616 просмотров • 1 месяц назад