Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Developer Advocate Suraj Subramanian talks all about PyTorch with Pascal Hartig on the latest Meta Tech Podcast. Learn about the history of #PyTorch, how it became successful, and the importance of fostering a helpful community. Listen now:

25,544 Aufrufe • vor 3 Jahren •via X (Twitter)

1 Kommentare

Profilbild von Meta Open Source
Meta Open Sourcevor 1 Jahr

⚡️OSS Project Spotlight⚡️ Glean is a powerful code indexing system that enables efficient code search and analysis across large codebases. It helps devs understand code structure and dependencies, enhancing productivity and collaboration. For more, visit

Ähnliche Videos

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,135 Aufrufe • vor 1 Jahr