Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

The next chapter about transformers is up on YouTube, digging into the attention mechanism: The model works with vectors representing tokens (think words), and this is the mechanism that allows those vectors to take in meaning from context.

Grant Sanderson

444,986 subscribers

810,215 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Grant Sanderson

Grant Sandersonvor 2 Jahren

As a preview, here's a sped-up version of one of the core scenes, though admittedly it may not make full sense without narration.

Profilbild von arthur

arthurvor 2 Jahren

I thought the next episode would take months or not ever come out, mad props for putting it out so quickly and at such high quality

Profilbild von near

nearvor 2 Jahren

great visuals, just subscribed to the patreon!

Profilbild von Jon Masters 🏴‍☠️

Jon Masters 🏴‍☠️vor 2 Jahren

This video series is going to be regarded as seminal in the field for understanding

Profilbild von Krishna Kaasyap

Krishna Kaasyapvor 2 Jahren

You and @karpathy are teaching an entire generation of people who are coming into this field because of this Transformer driven GenAI boom! Thanks for being an awesome teacher to us Grant. 🙏🏻

Profilbild von LepetitRo

LepetitRovor 2 Jahren

Your videos are incredibly good, I wish they were available in other languages

Profilbild von Ra'uf

Ra'ufvor 2 Jahren

Is there any intentions to make a video or a series about fractals and about the Mandelbrot set?

Profilbild von Grant Sanderson

Grant Sandersonvor 2 Jahren

Here's one I made about holomorphic dynamics, which includes things like the Mandelbrot set:

Profilbild von K3ith.AI

K3ith.AIvor 2 Jahren

Amazing Grant! Tku and welcome back!!!🥳🤩🦾

Profilbild von Ramakant Soni

Ramakant Sonivor 2 Jahren

I am so messed up rn I can’t even watch this guys videos in one go 😢

Ähnliche Videos

Most don't know (1) how easy it is to invert embedding vectors back into sentences, (2) this is a perfect task text diffusion models. Here's a 78M parameter model and live demo that recovers 80% of tokens from Qwen3-Embedding and EmbeddingGemma vectors. Works even on multilingual input.

Most don't know (1) how easy it is to invert embedding vectors back into sentences, (2) this is a perfect task text diffusion models. Here's a 78M parameter model and live demo that recovers 80% of tokens from Qwen3-Embedding and EmbeddingGemma vectors. Works even on multilingual input.

Jina AI

12,977 Aufrufe • vor 4 Monaten

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,216 Aufrufe • vor 1 Jahr

The interesting thing about mixing rasters with vectors in Rive is that you can create very cool things. In this case the dome, eyes and nose of the skull are vectors. The skull and the base of the dome are rasters. With the vectors I can illuminate the edges of the eyes and nose, or animate them in a simpler way.

The interesting thing about mixing rasters with vectors in Rive is that you can create very cool things. In this case the dome, eyes and nose of the skull are vectors. The skull and the base of the dome are rasters. With the vectors I can illuminate the edges of the eyes and nose, or animate them in a simpler way.

JcToon

44,940 Aufrufe • vor 1 Jahr

[Self-Attention] by Hand ✍️ Self-attention is what enables LLMs to understand context. How does it work? This exercise demonstrates how to calculate a 6-3 attention head by hand. Note that if we have two instances of this, we get 6-6 attention (i.e., multi-head attention, n=2). -- 𝗚𝗼𝗮𝗹 -- Transform [6D Features 🟧] to [3D Attention Weighted Features 🟦] -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- [1] Given ↳ A set of 4 feature vectors (6-D): x1,x2,x3,x4 [2] Query, Key, Value ↳ Multiply features x's with linear transformation matrices WQ, WK, and WV, to obtain query vectors (q1,q2,q3,q4), key vectors (k1,k2,k3,k4), and value vectors (v1,v2,v3,v4). ↳ "Self" refers to the fact that both queries and keys are derived from the same set of features. [3] 🟪 Prepare for MatMul ↳ Copy query vectors ↳ Copy the transpose of key vectors [4] 🟪 MatMul ↳ Multiply K^T and Q ↳ This is equivalent to taking dot product between every pair of query and key vectors. ↳ The purpose is to use dot product as an estimate of the "matching score" between every key-value pair. ↳ This estimate makes sense because dot product is the numerator of Cosine Similarity between two vectors. [5] 🟨 Scale ↳ Scale each element by the square root of dk, which is the dimension of key vectors (dk=3). ↳ The purpose is to normalize the impact of the dk on matching scores, even if we scale dk to 32, 64, or 128. ↳ To simplify hand calculation, we approximate [ □/sqrt(3) ] with [ floor(□/2) ]. [6] 🟩 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [7] 🟩 Softmax: ∑ ↳ Sum across each column [8] 🟩 Softmax: 1 / sum ↳ For each column, divide each element by the column sum ↳ The purpose is normalize each column so that the numbers sum to 1. In other words, each column is a probability distribution of attention, and we have four of them. ↳ The result is the Attention Weight Matrix (A) (yellow) [9] 🟦 MatMul ↳ Multiply the value vectors (Vs) with the Attention Weight Matrix (A) ↳ The results are the attention weighted features Zs. ↳ They are fed to the position-wise feed forward network in the next layer.

[Self-Attention] by Hand ✍️ Self-attention is what enables LLMs to understand context. How does it work? This exercise demonstrates how to calculate a 6-3 attention head by hand. Note that if we have two instances of this, we get 6-6 attention (i.e., multi-head attention, n=2). -- 𝗚𝗼𝗮𝗹 -- Transform [6D Features 🟧] to [3D Attention Weighted Features 🟦] -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- [1] Given ↳ A set of 4 feature vectors (6-D): x1,x2,x3,x4 [2] Query, Key, Value ↳ Multiply features x's with linear transformation matrices WQ, WK, and WV, to obtain query vectors (q1,q2,q3,q4), key vectors (k1,k2,k3,k4), and value vectors (v1,v2,v3,v4). ↳ "Self" refers to the fact that both queries and keys are derived from the same set of features. [3] 🟪 Prepare for MatMul ↳ Copy query vectors ↳ Copy the transpose of key vectors [4] 🟪 MatMul ↳ Multiply K^T and Q ↳ This is equivalent to taking dot product between every pair of query and key vectors. ↳ The purpose is to use dot product as an estimate of the "matching score" between every key-value pair. ↳ This estimate makes sense because dot product is the numerator of Cosine Similarity between two vectors. [5] 🟨 Scale ↳ Scale each element by the square root of dk, which is the dimension of key vectors (dk=3). ↳ The purpose is to normalize the impact of the dk on matching scores, even if we scale dk to 32, 64, or 128. ↳ To simplify hand calculation, we approximate [ □/sqrt(3) ] with [ floor(□/2) ]. [6] 🟩 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [7] 🟩 Softmax: ∑ ↳ Sum across each column [8] 🟩 Softmax: 1 / sum ↳ For each column, divide each element by the column sum ↳ The purpose is normalize each column so that the numbers sum to 1. In other words, each column is a probability distribution of attention, and we have four of them. ↳ The result is the Attention Weight Matrix (A) (yellow) [9] 🟦 MatMul ↳ Multiply the value vectors (Vs) with the Attention Weight Matrix (A) ↳ The results are the attention weighted features Zs. ↳ They are fed to the position-wise feed forward network in the next layer.

Tom Yeh

101,010 Aufrufe • vor 2 Jahren

Tom Mueller: “Elon was always about making all the vectors – the employees – pointing in the right direction: Forward.” “He was technically very strong even though he was just learning about rockets. And the one thing I noticed, if people were negative, they were not in the next meeting. He said, a company is a bunch of vectors. Each person is a vector and they need a point in the direction that you want to go. Bureaucracy and office politics and low morale, it's almost random vectors. He was always about making all the vectors which is all the employees, pointing in the right direction: Forward, moving forward.” From: The Elon Musk Show, 2022

Tom Mueller: “Elon was always about making all the vectors – the employees – pointing in the right direction: Forward.” “He was technically very strong even though he was just learning about rockets. And the one thing I noticed, if people were negative, they were not in the next meeting. He said, a company is a bunch of vectors. Each person is a vector and they need a point in the direction that you want to go. Bureaucracy and office politics and low morale, it's almost random vectors. He was always about making all the vectors which is all the employees, pointing in the right direction: Forward, moving forward.” From: The Elon Musk Show, 2022

ELON CLIPS

238,866 Aufrufe • vor 1 Jahr

AI can turn images into vectors! Layered Vectorization can turn images into layered vectors that represent the original image from coarse to fine detail levels. Links ⬇️

AI can turn images into vectors! Layered Vectorization can turn images into layered vectors that represent the original image from coarse to fine detail levels. Links ⬇️

Dreaming Tulpa 🥓👑

18,837 Aufrufe • vor 2 Jahren

Tom Mueller on SpaceX's early days: “If people were negative, they were not in the next meeting.” “Elon was technically very strong, even though he was just learning about rockets. The one thing I noticed, if people were negative, they were not in the next meeting. He said, a company is a bunch of vectors. Each person is a vector and they need to point in the direction that you want to go. Bureaucracy and office politics and low morale, it's almost random vectors. He was always about making all the vectors which are all the employees pointing in the right direction. Forward, moving forward.” From BBC's documentary The Elon Musk Show, 2022

Tom Mueller on SpaceX's early days: “If people were negative, they were not in the next meeting.” “Elon was technically very strong, even though he was just learning about rockets. The one thing I noticed, if people were negative, they were not in the next meeting. He said, a company is a bunch of vectors. Each person is a vector and they need to point in the direction that you want to go. Bureaucracy and office politics and low morale, it's almost random vectors. He was always about making all the vectors which are all the employees pointing in the right direction. Forward, moving forward.” From BBC's documentary The Elon Musk Show, 2022

ELON CLIPS

523,718 Aufrufe • vor 7 Monaten

How good a 2D rig can look? Darcy and Jarrod created this rough animation in Moho and then used vectors and brushes to transform it into an impressive live painting! “I’m impressed with just how much you can manipulate the vectors in Moho. The best part is it's fun!”

How good a 2D rig can look? Darcy and Jarrod created this rough animation in Moho and then used vectors and brushes to transform it into an impressive live painting! “I’m impressed with just how much you can manipulate the vectors in Moho. The best part is it's fun!”

Moho Animation Software

26,243 Aufrufe • vor 1 Jahr

‘The Antikythera Mechanism” lyric video is up on our Youtube Channel. How much do you like this song ?

‘The Antikythera Mechanism” lyric video is up on our Youtube Channel. How much do you like this song ?

Nightwish

12,383 Aufrufe • vor 1 Jahr

EARLY SPACEX EMPLOYEE GIVES KEY INSIGHT INTO ELON’S RELENTLESS PUSH FORWARD Tom Mueller, SpaceX's employee No. 1: “Elon was technically very strong, even though he was just learning about rockets. The one thing I noticed, if people were negative, they were not in the next meeting. He said, a company is a bunch of vectors. Each person is a vector, and they need to point in the direction that you want to go. Bureaucracy and office politics and low morale—they're almost random vectors. He was always about making all the vectors, which are all the employees pointing in the right direction. Forward, moving forward.” From BBC's documentary The Elon Musk Show, 2022

EARLY SPACEX EMPLOYEE GIVES KEY INSIGHT INTO ELON’S RELENTLESS PUSH FORWARD Tom Mueller, SpaceX's employee No. 1: “Elon was technically very strong, even though he was just learning about rockets. The one thing I noticed, if people were negative, they were not in the next meeting. He said, a company is a bunch of vectors. Each person is a vector, and they need to point in the direction that you want to go. Bureaucracy and office politics and low morale—they're almost random vectors. He was always about making all the vectors, which are all the employees pointing in the right direction. Forward, moving forward.” From BBC's documentary The Elon Musk Show, 2022

Mario Nawfal

774,211 Aufrufe • vor 7 Monaten

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

Avi Chawla

265,992 Aufrufe • vor 12 Tagen

Why I think the-open-network:native is being overlooked I feel like it’s one of those tokens that will do the rounds in the next bull market. Lmk your take on this

Why I think the-open-network:native is being overlooked I feel like it’s one of those tokens that will do the rounds in the next bull market. Lmk your take on this

Jonah

11,091 Aufrufe • vor 12 Tagen

【告知/Notice】【CF21 OPEN PO ANNOUNCEMENT】【Vital 7 Vectors】XFD Vital 7 Vectors combined 7 Composers and 7 Illustrators into one album! For those who are interested in purchasing the Physical Album please visit this link below! The album will set to release on 15 November!

【告知/Notice】【CF21 OPEN PO ANNOUNCEMENT】【Vital 7 Vectors】XFD Vital 7 Vectors combined 7 Composers and 7 Illustrators into one album! For those who are interested in purchasing the Physical Album please visit this link below! The album will set to release on 15 November!

ZxNX

19,804 Aufrufe • vor 8 Monaten

Moho as a cleanup tool? Darcy Woodbridge took his beautiful rough and transformed it into a cool animated illustration using vectors and brushes. It is all rigged! “I’m impressed with just how much you can manipulate the vectors in Moho. And the best part is it's fun!” #mohoanimation

Moho as a cleanup tool? Darcy Woodbridge took his beautiful rough and transformed it into a cool animated illustration using vectors and brushes. It is all rigged! “I’m impressed with just how much you can manipulate the vectors in Moho. And the best part is it's fun!” #mohoanimation

Moho Animation Software

210,073 Aufrufe • vor 2 Jahren

Last week, we launched "Attention in Transformers: Concepts and Code in PyTorch" instructed by Joshua Starmer! In this course, you'll: ✅ Learn how the attention mechanism in LLMs helps convert base token embeddings into rich context-aware embeddings. ✅ Understand the Query, Key, and Value matrices, what they are for, how to produce them, and how to use them in attention. ✅ Learn the difference between self-attention, masked self-attention, and cross-attention, and how multi-head attention scales the algorithm. 🔗 Enroll for free:

Last week, we launched "Attention in Transformers: Concepts and Code in PyTorch" instructed by Joshua Starmer! In this course, you'll: ✅ Learn how the attention mechanism in LLMs helps convert base token embeddings into rich context-aware embeddings. ✅ Understand the Query, Key, and Value matrices, what they are for, how to produce them, and how to use them in attention. ✅ Learn the difference between self-attention, masked self-attention, and cross-attention, and how multi-head attention scales the algorithm. 🔗 Enroll for free:

DeepLearning.AI

36,832 Aufrufe • vor 1 Jahr

Attention is a lookup. Each token builds a query, compares it against every key in the sequence, and pulls value vectors weighted by the match. Stack that 96 layers deep and you get a frontier model. Video covers the full pipeline: Q/K/V, attention scores, encoder blocks.

Attention is a lookup. Each token builds a query, compares it against every key in the sequence, and pulls value vectors weighted by the match. Stack that 96 layers deep and you get a frontier model. Video covers the full pipeline: Q/K/V, attention scores, encoder blocks.

tetsuo

103,177 Aufrufe • vor 7 Tagen

🇸🇦🇺🇸 DAVID SACKS: MCP IS THE NEXT BIG WAVE IN AI "MCP {Model Context Protocol} is a new standard that allows agents to talk to existing SaaS applications or any kind of application, and it's really going to enable this agentic future that I think is the next big wave in AI." Source: David Sacks

🇸🇦🇺🇸 DAVID SACKS: MCP IS THE NEXT BIG WAVE IN AI "MCP {Model Context Protocol} is a new standard that allows agents to talk to existing SaaS applications or any kind of application, and it's really going to enable this agentic future that I think is the next big wave in AI." Source: David Sacks

Mario Nawfal

123,166 Aufrufe • vor 1 Jahr

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: • In-Context Learning • Indexing + In-Context Learning • Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: • In-Context Learning • Indexing + In-Context Learning • Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

Santiago

384,495 Aufrufe • vor 3 Jahren

In our latest benchmark, a traditional text-to-SQL approach required more than 4,400 schema-related tokens per query just to provide the model with enough context to understand a 28-table insurance data model. Think about that for a second: before AI can interpret the question, it first has to read the instruction manual. What we found is that when business context is already organized and governed, AI doesn't need to repeatedly rediscover how your data works. We created this white paper to help companies direct their attention to where costs are piling up and how Mosaic directly reduces them. Read the benchmark here: #EnterpriseAI #DataAnalytics #SemanticLayer #BusinessIntelligence

In our latest benchmark, a traditional text-to-SQL approach required more than 4,400 schema-related tokens per query just to provide the model with enough context to understand a 28-table insurance data model. Think about that for a second: before AI can interpret the question, it first has to read the instruction manual. What we found is that when business context is already organized and governed, AI doesn't need to repeatedly rediscover how your data works. We created this white paper to help companies direct their attention to where costs are piling up and how Mosaic directly reduces them. Read the benchmark here: #EnterpriseAI #DataAnalytics #SemanticLayer #BusinessIntelligence

Strategy

11,097 Aufrufe • vor 22 Tagen

Idefics3-Llama is out! 💥 It's a multimodal model based on Llama 3.1 that accepts arbitrary number of interleaved images with text with a huge context window (10k tokens!) 😍 Link to demo and model in the next one 😏

Idefics3-Llama is out! 💥 It's a multimodal model based on Llama 3.1 that accepts arbitrary number of interleaved images with text with a huge context window (10k tokens!) 😍 Link to demo and model in the next one 😏

merve

28,014 Aufrufe • vor 1 Jahr