Loading video...

Video Failed to Load

Go Home

the transformer architecture in full detail

Hesamation's profile picture

ℏεsam

82,388 subscribers

113,289 views • 1 year ago •via X (Twitter)

Science & TechnologyNews & PoliticsEducation
LIVE

Anya Rossi• Live Now

Private livecam show

2 Comments

ℏεsam's profile picture
ℏεsam1 year ago

source:

Jash Sayani's profile picture
Jash Sayani1 year ago

Attention is all you need

Related Videos

Proper Terrace Drainage & Threshold Detail 🏗 #Architecture #Design
0:23

Proper Terrace Drainage & Threshold Detail 🏗 #Architecture #Design

Dòkun📍

825,214 views • 7 days ago

A masterpiece is not designed in panic. Great architecture needs time, discipline, and obsession with detail.
2:06

A masterpiece is not designed in panic. Great architecture needs time, discipline, and obsession with detail.

̊M̊̊Å̊V̊̊O̊̊R̊̊Y̊

12,927 views • 3 months ago

As promised LUXOR, THE SOURCE OF LIGHT - WILLIAM COOPER Full documentary 1994 William Cooper visited the Luxor Hotel in Las Vegas and explains in detail, the hidden occult symbolism intertwined within the architecture, letting you know who this city belongs to.
54:49

As promised LUXOR, THE SOURCE OF LIGHT - WILLIAM COOPER Full documentary 1994 William Cooper visited the Luxor Hotel in Las Vegas and explains in detail, the hidden occult symbolism intertwined within the architecture, letting you know who this city belongs to.

Redpill Drifter

247,885 views • 6 months ago

😀😀A short video to help you understand the Hybrid Mamba Transformer architecture behind the Hunyuan T1 & Turbo S models. Produced by EyeOpenAI.
5:19

😀😀A short video to help you understand the Hybrid Mamba Transformer architecture behind the Hunyuan T1 & Turbo S models. Produced by EyeOpenAI.

Tencent HY

21,922 views • 1 year ago

American architecture is so underrated. These homes have so much character and detail.
1:27

American architecture is so underrated. These homes have so much character and detail.

Financial Dystopia

1,021,507 views • 8 months ago

Transformer by hand ✍️ in Excel ~ I just released my first-ever "Full-Stack" implementation of the Transformer model. 👇Download xlsx to give it a try!
1:02

Transformer by hand ✍️ in Excel ~ I just released my first-ever "Full-Stack" implementation of the Transformer model. 👇Download xlsx to give it a try!

Tom Yeh

2,998,207 views • 1 year ago

Giovani Magana breaks down modular environment building for game-ready architecture. Learn how to assemble tileable assets into efficient chunks that scale without sacrificing detail. The full workflow is streaming now!
1:11

Giovani Magana breaks down modular environment building for game-ready architecture. Learn how to assemble tileable assets into efficient chunks that scale without sacrificing detail. The full workflow is streaming now!

The Gnomon Workshop

32,148 views • 7 months ago

I've asked Astra to read the DeepSeek v4.1 Flash paper and compare it to the original Transformer architecture in 3D - you can zoom in an inspect each element side by side. Things have changed quite a bit. Try yourself:
0:39

I've asked Astra to read the DeepSeek v4.1 Flash paper and compare it to the original Transformer architecture in 3D - you can zoom in an inspect each element side by side. Things have changed quite a bit. Try yourself:

Peter Gostev

269,611 views • 3 days ago

The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. Lukasz Kaiser came to defend the Transformer. Adrian Kosowski, Llion Jones, and Mathias Lechner made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.
1:20:46

The full Transformer vs Post-Transformer debate is live. 80 minutes. Seven rounds. No slides. Real disagreement. Lukasz Kaiser came to defend the Transformer. Adrian Kosowski, Llion Jones, and Mathias Lechner made the case for what comes next. 00:00 Contenders enter the ring 06:30 Lukasz Kaiser defends the Transformer 10:08 Adrian Kosowski on BDH and the PageRank Moment for AI 17:35 Llion Jones: Why Transformers aren't the final architecture 29:50 Mathias Lechner on Liquid AI’s approach, Fast Weights, and Self-Replacing AI 40:28 Reasoning Beyond Language 44:15 Scaling Laws: Transformer vs Post Transformer 50:31 Benchmarks, Coding Models, and Perplexity 1:04:00 Continual Learning and Dynamic Weights This is the ultimate source of truth on the subject.

Zuzanna Stamirowska

1,378,176 views • 3 months ago

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:
3:36

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:

Andrew Ng

259,920 views • 1 year ago

Homelander costume in detail watch the full video in the link below (via therealadamsavage/YT)
4:18

Homelander costume in detail watch the full video in the link below (via therealadamsavage/YT)

antonystarrgifs

131,347 views • 3 months ago

In 2017, the team who submitted the Transformer architecture was hoping for "hundreds of citations" 281,654 citations later, people are pointing to that paper as the catalyst for the next era of an industrial revolution. In Aidan Gomez's words, it was a 'productive 4 months':
1:21

In 2017, the team who submitted the Transformer architecture was hoping for "hundreds of citations" 281,654 citations later, people are pointing to that paper as the catalyst for the next era of an industrial revolution. In Aidan Gomez's words, it was a 'productive 4 months':

Cohere

34,831 views • 12 days ago

MOSS-TTS Local Transformer v1.5 is here. Clone any voice. Speak any language. Hear every detail. 30+ languages, 48 kHz clarity.
0:20

MOSS-TTS Local Transformer v1.5 is here. Clone any voice. Speak any language. Hear every detail. 30+ languages, 48 kHz clarity.

MOSI

122,335 views • 2 months ago

Sergey Brin admits Google messed up by under-investing in the transformer architecture it invented Google was too scared to release chatbots that "say dumb things", so it under-invested in scaling compute "we didn't take it very seriously... and openAI ran with it"
0:52

Sergey Brin admits Google messed up by under-investing in the transformer architecture it invented Google was too scared to release chatbots that "say dumb things", so it under-invested in scaling compute "we didn't take it very seriously... and openAI ran with it"

Haider.

1,264,327 views • 9 months ago

Cute Transformer in the new Kamen Rider 💖
1:07

Cute Transformer in the new Kamen Rider 💖

PaperPlane

178,561 views • 1 year ago

RIDDLES IN STONE: THE SECRET MASONIC ARCHITECTURE OF WASHINGTON D.C. The Capitol, like all Washington DC, has been almost entirely designed by Freemasons and integrates within its architecture and artwork the “keys to the Craft”. Full Documentary
2:53:54

RIDDLES IN STONE: THE SECRET MASONIC ARCHITECTURE OF WASHINGTON D.C. The Capitol, like all Washington DC, has been almost entirely designed by Freemasons and integrates within its architecture and artwork the “keys to the Craft”. Full Documentary

Redpill Drifter

30,174 views • 2 years ago

The Transformer upgrade is currently in progress at Teshie
2:00

The Transformer upgrade is currently in progress at Teshie

we love ghana

96,783 views • 4 months ago