Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

THE BEST visual explainer of how information propagates through a transformer. If you want to have more than intuition about how the Transformer architecture is ruling the LLM world - open-source project explains everything about LLM Transformer Models! - A great resource for anyone looking to gain a deeper... show more

Rohan Paul

150,073 subscribers

106,880 views • 1 year ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Wow. This is one of the best interactive sites I’ve seen for learning how LLMs work! 🔥 It starts w/ a clear intro and guides you through every core component: from Embedding, Layer Norm, and Self-Attention to MLPs, Transformer blocks, Softmax, and Output layers. link in 🧵↓

Wow. This is one of the best interactive sites I’ve seen for learning how LLMs work! 🔥 It starts w/ a clear intro and guides you through every core component: from Embedding, Layer Norm, and Self-Attention to MLPs, Transformer blocks, Softmax, and Output layers. link in 🧵↓

Charly Wargnier

52,092 views • 1 year ago

Can we make Transformers better and more efficient for robot learning? Excited to introduce Body Transformer (BoT), an architecture that leverages robot embodiment in the attention mechanism, by treating it as a graph of sensors and actuators.

Can we make Transformers better and more efficient for robot learning? Excited to introduce Body Transformer (BoT), an architecture that leverages robot embodiment in the attention mechanism, by treating it as a graph of sensors and actuators.

Carlo Sferrazza

60,663 views • 1 year ago

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Yan Chen

33,255 views • 1 year ago

How do you like my transformer?

How do you like my transformer?

Ilya Dreamy

33,426 views • 4 months ago

The rise and fall of popular transformer architectures 📈 One pro of having a lot of transformer architectures implemented in transformers & hosted on the Hugging Face Hub is that one can see the evolution of popularity of different models 👀

The rise and fall of popular transformer architectures 📈 One pro of having a lot of transformer architectures implemented in transformers & hosted on the Hugging Face Hub is that one can see the evolution of popularity of different models 👀

Lysandre

31,730 views • 2 years ago

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

Tech with Mak

82,077 views • 3 months ago

We’re releasing model weights for our 8B- parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. Learn more about how Dynamic Byte Latent Transformer is paving the way for groundbreaking developments in the field of language modeling. Read the research paper, and download the model and code.

We’re releasing model weights for our 8B- parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. Learn more about how Dynamic Byte Latent Transformer is paving the way for groundbreaking developments in the field of language modeling. Read the research paper, and download the model and code.

AI at Meta

195,525 views • 1 year ago

Check this!! A 100% open-source toolkit to work with LLMs. Transformer Lab is an app to experiment with LLMs: - Train, fine-tune, or chat. - One-click LLM download (DeepSeek, Gemma, etc.) - Drag-n-drop UI for RAG. - Built-in logging, and more. 100% local!

Check this!! A 100% open-source toolkit to work with LLMs. Transformer Lab is an app to experiment with LLMs: - Train, fine-tune, or chat. - One-click LLM download (DeepSeek, Gemma, etc.) - Drag-n-drop UI for RAG. - Built-in logging, and more. 100% local!

Avi Chawla

74,778 views • 1 year ago

Part 2/30 of the LLM Series: RoPE (Rotary Position Embedding) How does a transformer know the difference between - "the dog bit the man" and "the man bit the dog"? The words are almost identical, but the meaning changes completely. RoPE encodes position as rotation, allowing transformers to understand relative order through geometry. Read more:

Part 2/30 of the LLM Series: RoPE (Rotary Position Embedding) How does a transformer know the difference between - "the dog bit the man" and "the man bit the dog"? The words are almost identical, but the meaning changes completely. RoPE encodes position as rotation, allowing transformers to understand relative order through geometry. Read more:

TensorTonic

11,766 views • 28 days ago

Get ready for the PDE-Transformer: our new NN architecture tailored to scientific tasks. It combines hierarchical processing (UDiT), scalability (SWin) and flexible conditioning mechanisms. The paper shows it outperforming existing SOTA architectures 😁

Get ready for the PDE-Transformer: our new NN architecture tailored to scientific tasks. It combines hierarchical processing (UDiT), scalability (SWin) and flexible conditioning mechanisms. The paper shows it outperforming existing SOTA architectures 😁

Thuerey Group at TUM

13,422 views • 11 months ago

Transformers & LLMs cheatsheets for Stanford's CME-295! Covering tokenization, self-attention, prompting, fine-tuning, LLM-as-a-judge, RAG, AI Agents, and reasoning models. 100% free and open-source.

Transformers & LLMs cheatsheets for Stanford's CME-295! Covering tokenization, self-attention, prompting, fine-tuning, LLM-as-a-judge, RAG, AI Agents, and reasoning models. 100% free and open-source.

Akshay 🚀

101,587 views • 1 year ago

Our AI financial agent now shows its thinking Much like how Perplexity shows its tasks Here is how it works: • we have two LLMs • LLM #1 receives raw user query • LLM #1 creates tasks to answer query • LLM #2 receives these tasks • LLM #2 uses tools to execute tasks • LLM #2 streams answer to you We can add a 3rd LLM to do the last step. This would let LLM #2 focus only on task execution. I will add this in a follow-up. The code is open source so you can see how everything is done.

Our AI financial agent now shows its thinking Much like how Perplexity shows its tasks Here is how it works: • we have two LLMs • LLM #1 receives raw user query • LLM #1 creates tasks to answer query • LLM #2 receives these tasks • LLM #2 uses tools to execute tasks • LLM #2 streams answer to you We can add a 3rd LLM to do the last step. This would let LLM #2 focus only on task execution. I will add this in a follow-up. The code is open source so you can see how everything is done.

virat

19,481 views • 1 year ago

The best Tesla costume. Transformer.

The best Tesla costume. Transformer.

Tesla Owners Silicon Valley

57,978 views • 7 months ago

Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer

Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer

Teddy

20,741 views • 4 months ago

Switti -- a new scale-wise transformer for text-to-image generation 🦾 🔥 Improved generation of fine-grained details. Outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.

Switti -- a new scale-wise transformer for text-to-image generation 🦾 🔥 Improved generation of fine-grained details. Outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.

Gradio

29,267 views • 1 year ago

THIS HAPPENED YESTERDAY IN ENUGU! Transformer vandalizer electrocuted while trying to steal from the transformer. 💔😭

THIS HAPPENED YESTERDAY IN ENUGU! Transformer vandalizer electrocuted while trying to steal from the transformer. 💔😭

Benkingsley Nwashara

151,086 views • 3 months ago

What if the @Cybertruck was a Transformer? Made using Seedance 2.0

What if the @Cybertruck was a Transformer? Made using Seedance 2.0

CHRIS FIRST

303,918 views • 3 months ago

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

AK

124,042 views • 1 year ago

feeling like a transformer in Puerto Rico

feeling like a transformer in Puerto Rico

KANDY MUSE

30,481 views • 2 months ago