Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

THE BEST visual explainer of how information propagates through a transformer. If you want to have more than intuition about how the Transformer architecture is ruling the LLM world - open-source project explains everything about LLM Transformer Models! - A great resource for anyone looking to gain a deeper... show more

Rohan Paul

150,798 subscribers

106,897 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie Nachrichten & Politik Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Wow. This is one of the best interactive sites I’ve seen for learning how LLMs work! 🔥 It starts w/ a clear intro and guides you through every core component: from Embedding, Layer Norm, and Self-Attention to MLPs, Transformer blocks, Softmax, and Output layers. link in 🧵↓

Wow. This is one of the best interactive sites I’ve seen for learning how LLMs work! 🔥 It starts w/ a clear intro and guides you through every core component: from Embedding, Layer Norm, and Self-Attention to MLPs, Transformer blocks, Softmax, and Output layers. link in 🧵↓

Charly Wargnier

52,092 Aufrufe • vor 1 Jahr

Transformer by hand✍️ Excel ~ I designed this exercise to show the core math of a Transformer model is to combine columns (attention), combine rows (feed forward), and repeat.👇Join the 'AI Math' community. 👇Download xlsx.

Transformer by hand✍️ Excel ~ I designed this exercise to show the core math of a Transformer model is to combine columns (attention), combine rows (feed forward), and repeat.👇Join the 'AI Math' community. 👇Download xlsx.

Tom Yeh

66,770 Aufrufe • vor 1 Jahr

Can we make Transformers better and more efficient for robot learning? Excited to introduce Body Transformer (BoT), an architecture that leverages robot embodiment in the attention mechanism, by treating it as a graph of sensors and actuators.

Can we make Transformers better and more efficient for robot learning? Excited to introduce Body Transformer (BoT), an architecture that leverages robot embodiment in the attention mechanism, by treating it as a graph of sensors and actuators.

Carlo Sferrazza

60,679 Aufrufe • vor 1 Jahr

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Transformer: Multi-Head Attention ~ Math vs Code 🔢💻 ~ I made this visualization to show you how to implement the multi-head attention math in PyTorch within 50 LoC. Multi-Head Attention is what makes the Transformer's performance outstanding. It captures and represents more diverse linguistic relationships and patterns, and attends to different learned input embedding spaces. The parallel computing design also makes the model more efficient.

Yan Chen

33,326 Aufrufe • vor 1 Jahr

How do you like my transformer?

How do you like my transformer?

Ilya Dreamy

33,426 Aufrufe • vor 4 Monaten

The rise and fall of popular transformer architectures 📈 One pro of having a lot of transformer architectures implemented in transformers & hosted on the Hugging Face Hub is that one can see the evolution of popularity of different models 👀

The rise and fall of popular transformer architectures 📈 One pro of having a lot of transformer architectures implemented in transformers & hosted on the Hugging Face Hub is that one can see the evolution of popularity of different models 👀

Lysandre

31,730 Aufrufe • vor 2 Jahren

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

🚨 This is the best way to learn how LLMs work. Interactive. 3D. Step-by-step. Covers: → Embedding → Layer Norm → Self-Attention → MLP → Transformer layers → Softmax → Output Stop reading papers. Start seeing. Link in comments. Save this immediately.

Tech with Mak

82,177 Aufrufe • vor 4 Monaten

We’re releasing model weights for our 8B- parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. Learn more about how Dynamic Byte Latent Transformer is paving the way for groundbreaking developments in the field of language modeling. Read the research paper, and download the model and code.

We’re releasing model weights for our 8B- parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. Learn more about how Dynamic Byte Latent Transformer is paving the way for groundbreaking developments in the field of language modeling. Read the research paper, and download the model and code.

AI at Meta

195,546 Aufrufe • vor 1 Jahr

Check this!! A 100% open-source toolkit to work with LLMs. Transformer Lab is an app to experiment with LLMs: - Train, fine-tune, or chat. - One-click LLM download (DeepSeek, Gemma, etc.) - Drag-n-drop UI for RAG. - Built-in logging, and more. 100% local!

Check this!! A 100% open-source toolkit to work with LLMs. Transformer Lab is an app to experiment with LLMs: - Train, fine-tune, or chat. - One-click LLM download (DeepSeek, Gemma, etc.) - Drag-n-drop UI for RAG. - Built-in logging, and more. 100% local!

Avi Chawla

74,778 Aufrufe • vor 1 Jahr

Part 2/30 of the LLM Series: RoPE (Rotary Position Embedding) How does a transformer know the difference between - "the dog bit the man" and "the man bit the dog"? The words are almost identical, but the meaning changes completely. RoPE encodes position as rotation, allowing transformers to understand relative order through geometry. Read more:

Part 2/30 of the LLM Series: RoPE (Rotary Position Embedding) How does a transformer know the difference between - "the dog bit the man" and "the man bit the dog"? The words are almost identical, but the meaning changes completely. RoPE encodes position as rotation, allowing transformers to understand relative order through geometry. Read more:

TensorTonic

11,766 Aufrufe • vor 1 Monat

Get ready for the PDE-Transformer: our new NN architecture tailored to scientific tasks. It combines hierarchical processing (UDiT), scalability (SWin) and flexible conditioning mechanisms. The paper shows it outperforming existing SOTA architectures 😁

Get ready for the PDE-Transformer: our new NN architecture tailored to scientific tasks. It combines hierarchical processing (UDiT), scalability (SWin) and flexible conditioning mechanisms. The paper shows it outperforming existing SOTA architectures 😁

Thuerey Group at TUM

13,422 Aufrufe • vor 1 Jahr

Transformers & LLMs cheatsheets for Stanford's CME-295! Covering tokenization, self-attention, prompting, fine-tuning, LLM-as-a-judge, RAG, AI Agents, and reasoning models. 100% free and open-source.

Transformers & LLMs cheatsheets for Stanford's CME-295! Covering tokenization, self-attention, prompting, fine-tuning, LLM-as-a-judge, RAG, AI Agents, and reasoning models. 100% free and open-source.

Akshay 🚀

101,598 Aufrufe • vor 1 Jahr

Our AI financial agent now shows its thinking Much like how Perplexity shows its tasks Here is how it works: • we have two LLMs • LLM #1 receives raw user query • LLM #1 creates tasks to answer query • LLM #2 receives these tasks • LLM #2 uses tools to execute tasks • LLM #2 streams answer to you We can add a 3rd LLM to do the last step. This would let LLM #2 focus only on task execution. I will add this in a follow-up. The code is open source so you can see how everything is done.

Our AI financial agent now shows its thinking Much like how Perplexity shows its tasks Here is how it works: • we have two LLMs • LLM #1 receives raw user query • LLM #1 creates tasks to answer query • LLM #2 receives these tasks • LLM #2 uses tools to execute tasks • LLM #2 streams answer to you We can add a 3rd LLM to do the last step. This would let LLM #2 focus only on task execution. I will add this in a follow-up. The code is open source so you can see how everything is done.

virat

19,481 Aufrufe • vor 1 Jahr

The best Tesla costume. Transformer.

The best Tesla costume. Transformer.

Tesla Owners Silicon Valley

57,978 Aufrufe • vor 8 Monaten

Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer

Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer Generative Pen-trained Transformer

Teddy

20,896 Aufrufe • vor 4 Monaten

Switti -- a new scale-wise transformer for text-to-image generation 🦾 🔥 Improved generation of fine-grained details. Outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.

Switti -- a new scale-wise transformer for text-to-image generation 🦾 🔥 Improved generation of fine-grained details. Outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.

Gradio

29,267 Aufrufe • vor 1 Jahr

THIS HAPPENED YESTERDAY IN ENUGU! Transformer vandalizer electrocuted while trying to steal from the transformer. 💔😭

THIS HAPPENED YESTERDAY IN ENUGU! Transformer vandalizer electrocuted while trying to steal from the transformer. 💔😭

Benkingsley Nwashara

151,086 Aufrufe • vor 3 Monaten

What if the @Cybertruck was a Transformer? Made using Seedance 2.0

What if the @Cybertruck was a Transformer? Made using Seedance 2.0

CHRIS FIRST

305,579 Aufrufe • vor 4 Monaten

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

AK

124,048 Aufrufe • vor 1 Jahr