正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Self-Attention vs Cross-Attention interactive diagram. Open

Tom Yeh

65,671 subscribers

17,417 次观看 • 24 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

1 条评论

Ahmad Mujtaba 的头像

Ahmad Mujtaba23 天前

This is great ❤️

相关视频

Last week, we launched "Attention in Transformers: Concepts and Code in PyTorch" instructed by Joshua Starmer! In this course, you'll: ✅ Learn how the attention mechanism in LLMs helps convert base token embeddings into rich context-aware embeddings. ✅ Understand the Query, Key, and Value matrices, what they are for, how to produce them, and how to use them in attention. ✅ Learn the difference between self-attention, masked self-attention, and cross-attention, and how multi-head attention scales the algorithm. 🔗 Enroll for free:

Last week, we launched "Attention in Transformers: Concepts and Code in PyTorch" instructed by Joshua Starmer! In this course, you'll: ✅ Learn how the attention mechanism in LLMs helps convert base token embeddings into rich context-aware embeddings. ✅ Understand the Query, Key, and Value matrices, what they are for, how to produce them, and how to use them in attention. ✅ Learn the difference between self-attention, masked self-attention, and cross-attention, and how multi-head attention scales the algorithm. 🔗 Enroll for free:

DeepLearning.AI

36,832 次观看 • 1 年前

Seedance 2.0 is impressive. But it's closed-source! Introducing our daVinci-MagiHuman — a single-stream 15B Transformer trained from scratch that jointly generates video + audio. No cross-attention. No multi-stream branches. Just self-attention. ⚡ 5s 1080p video in 38s on a single H100 🏆 80% win rate vs Ovi 1.1 | 60.9% vs LTX 2.3 (2,000 human comparisons) 🌍 6 languages 📦 Fully open-source Speed by simplicity. By @SII_GAIR × Sand.ai 📄 💻 🤗

Seedance 2.0 is impressive. But it's closed-source! Introducing our daVinci-MagiHuman — a single-stream 15B Transformer trained from scratch that jointly generates video + audio. No cross-attention. No multi-stream branches. Just self-attention. ⚡ 5s 1080p video in 38s on a single H100 🏆 80% win rate vs Ovi 1.1 | 60.9% vs LTX 2.3 (2,000 human comparisons) 🌍 6 languages 📦 Fully open-source Speed by simplicity. By @SII_GAIR × Sand.ai 📄 💻 🤗

Pengfei Liu

301,178 次观看 • 6 个月前

self-attention explained by 3blue1brown's new video

self-attention explained by 3blue1brown's new video

ℏεsam

48,349 次观看 • 1 年前

🚨 ATTENTION 🚨 GAVIN CROSS HAS NOW HOMERED IN FIVE STRAIGHT GAMES!

🚨 ATTENTION 🚨 GAVIN CROSS HAS NOW HOMERED IN FIVE STRAIGHT GAMES!

Omaha Storm Chasers

83,236 次观看 • 2 个月前

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,544 次观看 • 1 年前

Motions that grab attention. 📩 open for work!

Motions that grab attention. 📩 open for work!

ARIA

12,439 次观看 • 9 个月前

Motions that grab attention. 📩 open for work!

Motions that grab attention. 📩 open for work!

ARIA

16,450 次观看 • 10 个月前

Pretending to self-harm yourself in prison for attention. The lowest is what?

Sensitive content

Pretending to self-harm yourself in prison for attention. The lowest is what?

Boochie Myers🔪

94,245 次观看 • 1 年前

Pay deeper attention to the people around you, stop being so self absorbed.

Pay deeper attention to the people around you, stop being so self absorbed.

Robert Greene

223,238 次观看 • 3 年前

Nick Aldis (c) vs Cody Rhodes NWA World Heavyweight Championship All In 2018 🎶 Attention Attention - Shinedown 🎶 Cody Rhodes Nick Aldis

Nick Aldis (c) vs Cody Rhodes NWA World Heavyweight Championship All In 2018 🎶 Attention Attention - Shinedown 🎶 Cody Rhodes Nick Aldis

CV Highlights

104,958 次观看 • 2 个月前

Pay attention when people are offended by the cross. It’s really all you need to know.

Pay attention when people are offended by the cross. It’s really all you need to know.

TheLizVariant

29,775 次观看 • 8 个月前

A lady intentionally exposed herself while dancing. Attention-seeking or self-expression? What's your take?

A lady intentionally exposed herself while dancing. Attention-seeking or self-expression? What's your take?

Ella Nnabugo

16,494 次观看 • 3 个月前

Easily create image maps on WordPress with the Draw Attention plugin. Immerse your audience with interactive content! Visit the demo below.

Easily create image maps on WordPress with the Draw Attention plugin. Immerse your audience with interactive content! Visit the demo below.

WP Draw Attention

8,384,147 次观看 • 1 年前

Attention Innovators💡 Applications for National Startup Awards 2023 are open. 👉

Attention Innovators💡 Applications for National Startup Awards 2023 are open. 👉

Piyush Goyal

586,545 次观看 • 3 年前

Turn any GitHub repository into a visual treat! Simply replace "hub" with "diagram" in a GitHub URL and instantly view the entire codebase as interactive diagrams for easier understanding. 100% open-source.

Turn any GitHub repository into a visual treat! Simply replace "hub" with "diagram" in a GitHub URL and instantly view the entire codebase as interactive diagrams for easier understanding. 100% open-source.

Akshay 🚀

125,479 次观看 • 1 年前

Found my usual spot at Kings Cross… Only this time… I gave the table leg some attention 😏

Found my usual spot at Kings Cross… Only this time… I gave the table leg some attention 😏

Chloe ✨

11,738 次观看 • 1 个月前