Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Met a guy making $1.6 million/year as an LLM engineer. I asked him how he learned LLMs from scratch. He sent me the exact video that got him in. A 1 hour course on how LLMs actually work. He shows how transformers inside LLMs like ChatGPT & Claude are... show more

Roan

58,115 subscribers

73,925 views • 1 day ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

My friend applied to 200 tech jobs in two years. No PhD. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me a course that was never supposed to get out. A 3-hour video to build a full LLM from scratch. A developer teaches you exactly how LLMs like ChatGPT and Claude are actually built. I watched it last night. Halfway through, I realized it's embarrassingly simple to break into an AI lab. Bookmark this and read the article below. • 00:00 - intro to LLMs • 05:43 - LLM transformer architecture • 40:24 - training the LLM • 1:30:27 - modernizing the LLM • 2:33:53 - scaling the LLM

My friend applied to 200 tech jobs in two years. No PhD. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me a course that was never supposed to get out. A 3-hour video to build a full LLM from scratch. A developer teaches you exactly how LLMs like ChatGPT and Claude are actually built. I watched it last night. Halfway through, I realized it's embarrassingly simple to break into an AI lab. Bookmark this and read the article below. • 00:00 - intro to LLMs • 05:43 - LLM transformer architecture • 40:24 - training the LLM • 1:30:27 - modernizing the LLM • 2:33:53 - scaling the LLM

Roan

275,163 views • 18 days ago

My friend applied to 300 tech jobs in two years. No MIT. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me the exact video that got him in. Andrej Karpathy's 3-hour course on building a full LLM from scratch. Anthropic's own researcher shows you how LLMs like ChatGPT & Claude are actually built. I watched it last night. Halfway through, I realized it's embarrassingly simple to break into an AI lab. Bookmark this and read the article below. • 00:00 - intro to LLMs • 12:41 - LLM training pipeline • 31:58 - LLM tools & plugins • 41:14 - LLM transformer architecture • 2:19:02 - scaling the LLM

My friend applied to 300 tech jobs in two years. No MIT. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me the exact video that got him in. Andrej Karpathy's 3-hour course on building a full LLM from scratch. Anthropic's own researcher shows you how LLMs like ChatGPT & Claude are actually built. I watched it last night. Halfway through, I realized it's embarrassingly simple to break into an AI lab. Bookmark this and read the article below. • 00:00 - intro to LLMs • 12:41 - LLM training pipeline • 31:58 - LLM tools & plugins • 41:14 - LLM transformer architecture • 2:19:02 - scaling the LLM

Roan

280,667 views • 8 days ago

Met a guy making $1.4 million a year as a prompt engineer. I asked him how he learned prompting so well. He sent me a video that was never supposed to get out. Andrew Ng's 2 hour prompting course. You wont find anything better about prompting than this video. I watched it last night. Halfway through, I realized I have been using Claude completely wrong for years. Bookmark and watch this today.

Met a guy making $1.4 million a year as a prompt engineer. I asked him how he learned prompting so well. He sent me a video that was never supposed to get out. Andrew Ng's 2 hour prompting course. You wont find anything better about prompting than this video. I watched it last night. Halfway through, I realized I have been using Claude completely wrong for years. Bookmark and watch this today.

Roan

315,545 views • 28 days ago

My friend applied to 200 tech jobs in two years. No MIT. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me the exact video that got him in. A 4-hour course on mastering Claude Code. I watched it last night. Halfway through, I realized I've been using Claude Code completely wrong for a year. Bookmark this and read the article below. • 00:00 - Claude Code setup • 49:34 - building apps with Claude Code • 2:07:52 - prompting Claude Code • 2:46:16 - Claude Code for production

My friend applied to 200 tech jobs in two years. No MIT. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me the exact video that got him in. A 4-hour course on mastering Claude Code. I watched it last night. Halfway through, I realized I've been using Claude Code completely wrong for a year. Bookmark this and read the article below. • 00:00 - Claude Code setup • 49:34 - building apps with Claude Code • 2:07:52 - prompting Claude Code • 2:46:16 - Claude Code for production

Roan

37,290 views • 12 days ago

My friend makes $850,000/year as an Anthropic AI engineer. No MIT. No Stanford. No PhD. I asked him how he broke into AI from scratch. He sent me a course that was never supposed to get out. A developer teaching to build self-improving AI agents from scratch. You will learn exactly how AI agents like Claude Code and OpenClaw are actually built. You won't find anything better about becoming an AI engineer in 2026 than this video. I watched it last night. Halfway through, I realized I could land a role at a top AI lab in months, not years. Bookmark this and read the article below.

My friend makes $850,000/year as an Anthropic AI engineer. No MIT. No Stanford. No PhD. I asked him how he broke into AI from scratch. He sent me a course that was never supposed to get out. A developer teaching to build self-improving AI agents from scratch. You will learn exactly how AI agents like Claude Code and OpenClaw are actually built. You won't find anything better about becoming an AI engineer in 2026 than this video. I watched it last night. Halfway through, I realized I could land a role at a top AI lab in months, not years. Bookmark this and read the article below.

Roan

69,565 views • 17 days ago

My friend makes $1.2 million a year as an Anthropic engineer. I asked him how he learned prompting so well. He sent me a video that was never supposed to get out. Their core team's prompting playbook. You won’t find anything better about prompting than this video. I watched it last night. Halfway through, I realized I've been using Claude completely wrong for two years. Watch it, then read the article below.

My friend makes $1.2 million a year as an Anthropic engineer. I asked him how he learned prompting so well. He sent me a video that was never supposed to get out. Their core team's prompting playbook. You won’t find anything better about prompting than this video. I watched it last night. Halfway through, I realized I've been using Claude completely wrong for two years. Watch it, then read the article below.

Raytar

1,408,991 views • 28 days ago

building large language models from scratch by Sebastian Raschka was a great chance for me to sit down and study again all the LLM basics > token and positional embeddings > self-attention and what QKV is about > causal & multi-head attention studying llms and how they work can seem overwhelming at first. but once you taste how good it feels to learn these things intuitively there's no going back. I shared the resources and my notes on my repo and I hope it's a motivation if you want to start as well or recap.

building large language models from scratch by Sebastian Raschka was a great chance for me to sit down and study again all the LLM basics > token and positional embeddings > self-attention and what QKV is about > causal & multi-head attention studying llms and how they work can seem overwhelming at first. but once you taste how good it feels to learn these things intuitively there's no going back. I shared the resources and my notes on my repo and I hope it's a motivation if you want to start as well or recap.

ℏεsam

49,292 views • 1 year ago

My friend applied to 250 tech jobs in two years. No MIT. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me the exact video that got him in. Anthropic's 2-hour course on how to become an AI engineer in 2026. Thariq Shihipar shows you exactly how to build AI agents from scratch. I watched it last night. Halfway through, I realized I could break into an AI lab in months, not years. Bookmark this and read the article below. • 00:00 - AI agent harness • 23:44 - building AI agent loops • 56:39 - AI agent context engineering • 1:33:34 - AI agent deterministic hooks • 1:50:31 - Anthropic SWE interview process

My friend applied to 250 tech jobs in two years. No MIT. No Stanford. Last month Anthropic offered him $750,000. I asked him how he broke in from zero. He sent me the exact video that got him in. Anthropic's 2-hour course on how to become an AI engineer in 2026. Thariq Shihipar shows you exactly how to build AI agents from scratch. I watched it last night. Halfway through, I realized I could break into an AI lab in months, not years. Bookmark this and read the article below. • 00:00 - AI agent harness • 23:44 - building AI agent loops • 56:39 - AI agent context engineering • 1:33:34 - AI agent deterministic hooks • 1:50:31 - Anthropic SWE interview process

Roan

685,926 views • 5 days ago

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,198,485 views • 2 months ago

New course! Generative AI with Large Language Models, created with Amazon Web Services and hosted on Coursera. This course goes deep into the technical foundations of LLMs and how to use them. You can sign up here: You’ll work through the full life-cycle of a generative AI project, and learn specific techniques like RLHF; zero-shot, one-shot, and few-shot learning with LLMs; advanced prompting frameworks like ReAct; even fine-tuning LLMs, and gain hands-on practice with all of these techniques. Instructors Antje Barth Chris Fregly Shelbee Eigenbrode and Mike G Chambers all do incredible Generative AI work at AWS, and have supported many companies to build creative LLM applications. They bring tremendous practical LLM expertise to this course. I'm confident you’ll finish this course with a deeper understanding of how LLMs work, and how to use them. I hope you enjoy the course!

New course! Generative AI with Large Language Models, created with Amazon Web Services and hosted on Coursera. This course goes deep into the technical foundations of LLMs and how to use them. You can sign up here: You’ll work through the full life-cycle of a generative AI project, and learn specific techniques like RLHF; zero-shot, one-shot, and few-shot learning with LLMs; advanced prompting frameworks like ReAct; even fine-tuning LLMs, and gain hands-on practice with all of these techniques. Instructors Antje Barth Chris Fregly Shelbee Eigenbrode and Mike G Chambers all do incredible Generative AI work at AWS, and have supported many companies to build creative LLM applications. They bring tremendous practical LLM expertise to this course. I'm confident you’ll finish this course with a deeper understanding of how LLMs work, and how to use them. I hope you enjoy the course!

Andrew Ng

467,912 views • 3 years ago

A Stanford professor just gave a public lecture on exactly how GPT, Claude, and LLaMA are built under the hood no insider access required just the clearest breakdown of modern LLM architecture I've seen this lecture reveals the framework professors are paid up to $750K a year to teach the gap between "I use ChatGPT" and "I understand how it works" is smaller than most people think the most complete public breakdown of modern LLM architecture I've seen this year

A Stanford professor just gave a public lecture on exactly how GPT, Claude, and LLaMA are built under the hood no insider access required just the clearest breakdown of modern LLM architecture I've seen this lecture reveals the framework professors are paid up to $750K a year to teach the gap between "I use ChatGPT" and "I understand how it works" is smaller than most people think the most complete public breakdown of modern LLM architecture I've seen this year

rewind

162,870 views • 1 month ago

My friend makes $1.4 million/year as an Anthropic engineer. I asked him how he learned prompting so well. He sent me a video that was never supposed to get out. Their core team's prompting roundtable. You won’t find anything better about prompting than this video. I watched it last night. Halfway through, I realized I've been using Claude the wrong way for two years. Bookmark & watch this before someone takes it down.

My friend makes $1.4 million/year as an Anthropic engineer. I asked him how he learned prompting so well. He sent me a video that was never supposed to get out. Their core team's prompting roundtable. You won’t find anything better about prompting than this video. I watched it last night. Halfway through, I realized I've been using Claude the wrong way for two years. Bookmark & watch this before someone takes it down.

Roan

81,142 views • 25 days ago

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!

Andrew Ng

502,821 views • 2 years ago

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,220 views • 1 year ago

A quick video about how Come-from-Beyond discovered that you can actually break any Large Language Model by trolling it with complex questions. It is called a "Zero Delta" exploit and all LLM models are susceptible to it. I managed to recreate this on Grok and the video shows the result. With regards to all LLMs out there from $QUBIC - building the real AGI. Qubic My YouTube Channel:

A quick video about how Come-from-Beyond discovered that you can actually break any Large Language Model by trolling it with complex questions. It is called a "Zero Delta" exploit and all LLM models are susceptible to it. I managed to recreate this on Grok and the video shows the result. With regards to all LLMs out there from $QUBIC - building the real AGI. Qubic My YouTube Channel:

retrodrive ⛏

13,625 views • 1 year ago

New short course on Reinforcement Learning from Human Feedback! RLHF is one of the key techniques that led to the rise of modern LLMs. It is used to align LLMs with human preferences, to make them more honest, helpful and harmless, by (i) learning a reward function that mimics human preferences, as expressed in human-provided labels, then, (ii) tuning an LLM to generate outputs that receive a high reward. In this course, taught by Nikita Namjoshi, Developer Advocate for GenAI at Google Cloud, you'll learn the details of how RLHF works, including how to apply it to tune an LLM for your own applications. You'll also use an open source library to tune a base LLM to align with human preferences expressed in a training set, and evaluate the tuned model by comparing its responses before and after RLHF-tuning. Please sign up here!

New short course on Reinforcement Learning from Human Feedback! RLHF is one of the key techniques that led to the rise of modern LLMs. It is used to align LLMs with human preferences, to make them more honest, helpful and harmless, by (i) learning a reward function that mimics human preferences, as expressed in human-provided labels, then, (ii) tuning an LLM to generate outputs that receive a high reward. In this course, taught by Nikita Namjoshi, Developer Advocate for GenAI at Google Cloud, you'll learn the details of how RLHF works, including how to apply it to tune an LLM for your own applications. You'll also use an open source library to tune a base LLM to align with human preferences expressed in a training set, and evaluate the tuned model by comparing its responses before and after RLHF-tuning. Please sign up here!

Andrew Ng

205,542 views • 2 years ago

.Andrej Karpathy says that LLMs currently lack the cultural accumulation and self-play that propelled humans out of the savannah: Culture: > “Why can’t an LLM write a book for the other LLMs? Why can’t other LLMs read this LLM’s book and be inspired by it, or shocked by it?” Self play: > “It’s extremely powerful. Evolution has a lot of competition driving intelligence and evolution. AlphaGo is playing against itself and that’s how it learns to get really good at Go. There’s no equivalent of self-play in LLMs. Why can’t an LLM, for example, create a bunch of problems that another LLM is learning to solve? Then the LLM is always trying to serve more and more difficult problems.” I asked Karpathy why LLMs still aren't yet able to build up culture the way humans do. > “The dumber models remarkably resemble a kindergarten student. [The smartest models still feel like] elementary school students though. Somehow, we still haven’t graduated enough where [these models] can take over. My Claude Code or Codex, they still feel like this elementary-grade student. I know that they can take PhD quizzes, but they still cognitively feel like a kindergarten.” > “I don’t think they can create culture because they’re still kids. They’re savant kids. They have perfect memory. They can convincingly create all kinds of slop that looks really good. But I still think they don’t really know what they’re doing. They don’t really have the cognition across all these little checkboxes that we still have to collect.”

.Andrej Karpathy says that LLMs currently lack the cultural accumulation and self-play that propelled humans out of the savannah: Culture: > “Why can’t an LLM write a book for the other LLMs? Why can’t other LLMs read this LLM’s book and be inspired by it, or shocked by it?” Self play: > “It’s extremely powerful. Evolution has a lot of competition driving intelligence and evolution. AlphaGo is playing against itself and that’s how it learns to get really good at Go. There’s no equivalent of self-play in LLMs. Why can’t an LLM, for example, create a bunch of problems that another LLM is learning to solve? Then the LLM is always trying to serve more and more difficult problems.” I asked Karpathy why LLMs still aren't yet able to build up culture the way humans do. > “The dumber models remarkably resemble a kindergarten student. [The smartest models still feel like] elementary school students though. Somehow, we still haven’t graduated enough where [these models] can take over. My Claude Code or Codex, they still feel like this elementary-grade student. I know that they can take PhD quizzes, but they still cognitively feel like a kindergarten.” > “I don’t think they can create culture because they’re still kids. They’re savant kids. They have perfect memory. They can convincingly create all kinds of slop that looks really good. But I still think they don’t really know what they’re doing. They don’t really have the cognition across all these little checkboxes that we still have to collect.”

Dwarkesh Patel

261,224 views • 9 months ago

Everyone is focused on tracking the ways LLMs are getting better. And they are. But we know there are still things that LLMs can’t do well—the tasks where you can feel the architecture fighting the problem. So I was excited to chat with Eve Bodnia (@eve_bodnia), who is developing an alternative AI model to LLMs, on Every 📧's AI & I. Eve's argument: energy-based models (EBMs), which map possible outcomes onto a mathematical landscape, will lead to the next AI phase shift. We get into: - How energy-based models work. Likely outcomes sit in valleys, and unlikely ones sit on peaks. Whereas LLMs process one token at a time, an EBM scans the full terrain to find the lowest point, or the most probable answer. - Language-based versus data-native models. LLMs are language-dependent even when the problem has nothing to do with language. "If your data is numbers, relationships, and functions, and you try to map those rules into words and then search for the next word, you're losing a lot of information," Bodnia says. EBMs work directly with the underlying data structure, including numbers and spatial coordinates. - Sequential versus panoramic reasoning. An LLM is like driving through San Francisco without a map. Each turn constrains the next, and if you go down the wrong street, you can't reverse course. An EBM has the bird's-eye view—it can evaluate multiple routes at once and course-correct before hitting a dead end. - The LLM plateau no one wants to talk about. LLMs are getting incrementally better, step-change improvements aren’t coming, Eve argues. To achieve that, we need new solutions that compensate for what LLMs are inherently bad at, like non-language reasoning, verification, and real-time data analysis. This is a must-watch for anyone who's curious what might come after the LLM. Watch below! Timestamps: Introduction: 00:00:51 Why correctness and verifiability matter in AI: 00:02:09 What an energy-based model is: 00:09:33 How EBMs construct energy landscapes to understand data: 00:14:21 Why modeling intelligence through language alone is a flawed approach: 00:19:00 What it means for a model to "understand" data: 00:26:54 How EBMs solve the vibe coding problem and enable formally verified code: 00:37:21 Why LLM progress is plateauing: 00:43:21 Mission-critical industries haven't adopted LLMs, and why EBMs can fill that gap: 00:49:54

Everyone is focused on tracking the ways LLMs are getting better. And they are. But we know there are still things that LLMs can’t do well—the tasks where you can feel the architecture fighting the problem. So I was excited to chat with Eve Bodnia (@eve_bodnia), who is developing an alternative AI model to LLMs, on Every 📧's AI & I. Eve's argument: energy-based models (EBMs), which map possible outcomes onto a mathematical landscape, will lead to the next AI phase shift. We get into: - How energy-based models work. Likely outcomes sit in valleys, and unlikely ones sit on peaks. Whereas LLMs process one token at a time, an EBM scans the full terrain to find the lowest point, or the most probable answer. - Language-based versus data-native models. LLMs are language-dependent even when the problem has nothing to do with language. "If your data is numbers, relationships, and functions, and you try to map those rules into words and then search for the next word, you're losing a lot of information," Bodnia says. EBMs work directly with the underlying data structure, including numbers and spatial coordinates. - Sequential versus panoramic reasoning. An LLM is like driving through San Francisco without a map. Each turn constrains the next, and if you go down the wrong street, you can't reverse course. An EBM has the bird's-eye view—it can evaluate multiple routes at once and course-correct before hitting a dead end. - The LLM plateau no one wants to talk about. LLMs are getting incrementally better, step-change improvements aren’t coming, Eve argues. To achieve that, we need new solutions that compensate for what LLMs are inherently bad at, like non-language reasoning, verification, and real-time data analysis. This is a must-watch for anyone who's curious what might come after the LLM. Watch below! Timestamps: Introduction: 00:00:51 Why correctness and verifiability matter in AI: 00:02:09 What an energy-based model is: 00:09:33 How EBMs construct energy landscapes to understand data: 00:14:21 Why modeling intelligence through language alone is a flawed approach: 00:19:00 What it means for a model to "understand" data: 00:26:54 How EBMs solve the vibe coding problem and enable formally verified code: 00:37:21 Why LLM progress is plateauing: 00:43:21 Mission-critical industries haven't adopted LLMs, and why EBMs can fill that gap: 00:49:54

Dan Shipper 📧

26,900 views • 3 months ago

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:

Announcing How Transformer LLMs Work, created with Jay Alammar and Maarten Grootendorst, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.” This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs). The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek. In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components. In details, you’ll learn: - How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input. - How inputs are broken down into tokens before they are sent to the language model. - The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head. - The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training. - How cached calculations make transformers faster. - Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs. By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details. Gaining this intuition will improve your approach to building LLM applications. Please sign up here:

Andrew Ng

259,421 views • 1 year ago

My friend makes $1.4 million/year as a Jane Street quant. No MIT. No Stanford. No Ivy League PhD. I asked him how he broke into quant from scratch. He sent me a video that was never supposed to get out. Hedge fund insiders laying out the full quant career playbook. You won't find anything better about becoming a quant than this video. I watched it last night. Halfway through, I realized I could break into a top quant firm in months, not years. Bookmark this and read the article below.

My friend makes $1.4 million/year as a Jane Street quant. No MIT. No Stanford. No Ivy League PhD. I asked him how he broke into quant from scratch. He sent me a video that was never supposed to get out. Hedge fund insiders laying out the full quant career playbook. You won't find anything better about becoming a quant than this video. I watched it last night. Halfway through, I realized I could break into a top quant firm in months, not years. Bookmark this and read the article below.

Roan

142,301 views • 20 days ago