正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

the first free interactive linear algebra textbook: - vectors and matrices - dot product and vector product - determinants - matrix ranks - eigenvalues and eigenvectors

ℏεsam

72,907 subscribers

63,641 次观看 • 1 年前 •via X (Twitter)

科学技术教育健康养生

Anya Rossi• Live Now

Private livecam show

4 条评论

ℏεsam 的头像

ℏεsam1 年前

🔗

Sayan Chakraborty 的头像

Sayan Chakraborty1 年前

damn, where was this when I was growing up. beautiful!

steve ike 的头像

steve ike1 年前

Love the idea of interactive textbooks.

Aditya Dixit 的头像

Aditya Dixit1 年前

Interactive math textbooks are the way to go! Thanks for sharing.

相关视频

🧵1/n This thread covers the linear algebra for modern computer graphics, bridging geometry, physics, and computation. Video 1: Vector Spaces & Cartesian Coordinates Source: CMU. 👇Video 2: Vector Operations, Vector Spaces, Functions as Vectors, and Vectors in Coordinates

🧵1/n This thread covers the linear algebra for modern computer graphics, bridging geometry, physics, and computation. Video 1: Vector Spaces & Cartesian Coordinates Source: CMU. 👇Video 2: Vector Operations, Vector Spaces, Functions as Vectors, and Vectors in Coordinates

tetsuo

52,043 次观看 • 1 年前

The Trap in Every Mathematics Lecture If you’ve taken enough math courses, you start noticing the same little move. The lecturer warms up with the obvious stuff, add matrices entrywise, scale by α, do the row-column product, and you’re thinking alright, where is this going. Then you relax. You stop resisting. And right there, they drop one line that quietly rewires the whole subject. When Benedict Gross says matrices represent linear operators, he’s telling you to stop treating a matrix as a rectangle of numbers and start treating it as an action. A linear operator is a function T: ℝⁿ → ℝⁿ that respects two rules: T(u+v) = T(u) + T(v) T(αu) = αT(u) Once you pick a basis, T is completely determined by where it sends the basis vectors e₁,…,eₙ. Put T(e₁),…,T(eₙ) into columns and you get a matrix A. That is what A represents T means. A is the coordinate portrait of the transformation. Now the punchline that makes matrix multiplication feel inevitable. If B represents S and A represents T, then doing S first and then T is the composition T∘S. In coordinates that becomes A(Bx) = (AB)x. So multiplying matrices is really composing transformations. That’s why multiplication is usually not commutative. T∘S is generally not the same transformation as S∘T, and the matrices inherit that noncommutativity. This explains half of linear algebra because it tells you what the course is really about: functions that move vectors around, not grids of numbers. A matrix is just the written form of that function once you choose coordinates. After that, the rules stop feeling random. Multiplying matrices means doing one move and then another. An inverse means you can undo the move. Eigenvectors are directions that don’t get turned. Changing basis is just describing the same move in a different language. One idea, and a lot of linear algebra suddenly clicks. #LinearAlgebra #Matrices #LinearMaps #Eigenvectors #ChangeOfBasis #Mathematics

The Trap in Every Mathematics Lecture If you’ve taken enough math courses, you start noticing the same little move. The lecturer warms up with the obvious stuff, add matrices entrywise, scale by α, do the row-column product, and you’re thinking alright, where is this going. Then you relax. You stop resisting. And right there, they drop one line that quietly rewires the whole subject. When Benedict Gross says matrices represent linear operators, he’s telling you to stop treating a matrix as a rectangle of numbers and start treating it as an action. A linear operator is a function T: ℝⁿ → ℝⁿ that respects two rules: T(u+v) = T(u) + T(v) T(αu) = αT(u) Once you pick a basis, T is completely determined by where it sends the basis vectors e₁,…,eₙ. Put T(e₁),…,T(eₙ) into columns and you get a matrix A. That is what A represents T means. A is the coordinate portrait of the transformation. Now the punchline that makes matrix multiplication feel inevitable. If B represents S and A represents T, then doing S first and then T is the composition T∘S. In coordinates that becomes A(Bx) = (AB)x. So multiplying matrices is really composing transformations. That’s why multiplication is usually not commutative. T∘S is generally not the same transformation as S∘T, and the matrices inherit that noncommutativity. This explains half of linear algebra because it tells you what the course is really about: functions that move vectors around, not grids of numbers. A matrix is just the written form of that function once you choose coordinates. After that, the rules stop feeling random. Multiplying matrices means doing one move and then another. An inverse means you can undo the move. Eigenvectors are directions that don’t get turned. Changing basis is just describing the same move in a different language. One idea, and a lot of linear algebra suddenly clicks. #LinearAlgebra #Matrices #LinearMaps #Eigenvectors #ChangeOfBasis #Mathematics

Mathelirium

133,454 次观看 • 5 个月前

The Trap in Every Mathematics Lecture If you’ve taken a lot of math courses, you start to recognize a pattern. There’s a moment where the lecturer is warming up with the obvious stuff...add matrices entrywise, scale by α, do the row-column product...and you’re thinking, alright… where is this going? Then you relax. You stop resisting. And right there, they slip in one line that changes how you see the whole subject. When Benedict Gross says "matrices represent linear operators,"he’s telling you to stop treating a matrix as a rectangle of numbers and start treating it as an action. A linear operator is a function T: Rⁿ → Rⁿ that respects two rules: T(u+v)=T(u)+T(v) and T(αu)=αT(u). Once you pick a basis, T is completely determined by where it sends the basis vectors e₁,…,eₙ. Put T(e₁),…,T(eₙ) into columns and you get a matrix A. That is what "A represents T" means...A is the coordinate portrait of the transformation. Now the punchline that makes matrix multiplication feel inevitable. If B represents S and A represents T, then doing S first and then T is the composition T∘S. In coordinates that becomes A(Bx)=(AB)x. So multiplying matrices is really composing transformations. That’s why multiplication is usually not commutative: T∘S is generally not the same transformation as S∘T, and the matrices inherit that noncommutativity. This explains half of Linear Algebra because it tells you what the course is really about...functions that move vectors around, not grids of numbers. A matrix is just the written form of that function once you choose coordinates. Then the rules stop feeling random Multiplying matrices means doing one move and then another, an inverse means you can undo the move, eigenvectors are directions that don’t get turned, and changing basis is just describing the same move in a different language. That one idea makes a lot of linear algebra click. #LinearAlgebra #Matrices #GroupTheory #GLn #MathLectures #Mathematics

The Trap in Every Mathematics Lecture If you’ve taken a lot of math courses, you start to recognize a pattern. There’s a moment where the lecturer is warming up with the obvious stuff...add matrices entrywise, scale by α, do the row-column product...and you’re thinking, alright… where is this going? Then you relax. You stop resisting. And right there, they slip in one line that changes how you see the whole subject. When Benedict Gross says "matrices represent linear operators,"he’s telling you to stop treating a matrix as a rectangle of numbers and start treating it as an action. A linear operator is a function T: Rⁿ → Rⁿ that respects two rules: T(u+v)=T(u)+T(v) and T(αu)=αT(u). Once you pick a basis, T is completely determined by where it sends the basis vectors e₁,…,eₙ. Put T(e₁),…,T(eₙ) into columns and you get a matrix A. That is what "A represents T" means...A is the coordinate portrait of the transformation. Now the punchline that makes matrix multiplication feel inevitable. If B represents S and A represents T, then doing S first and then T is the composition T∘S. In coordinates that becomes A(Bx)=(AB)x. So multiplying matrices is really composing transformations. That’s why multiplication is usually not commutative: T∘S is generally not the same transformation as S∘T, and the matrices inherit that noncommutativity. This explains half of Linear Algebra because it tells you what the course is really about...functions that move vectors around, not grids of numbers. A matrix is just the written form of that function once you choose coordinates. Then the rules stop feeling random Multiplying matrices means doing one move and then another, an inverse means you can undo the move, eigenvectors are directions that don’t get turned, and changing basis is just describing the same move in a different language. That one idea makes a lot of linear algebra click. #LinearAlgebra #Matrices #GroupTheory #GLn #MathLectures #Mathematics

Mathelirium

66,892 次观看 • 6 个月前

Pseudospectral Mirage of Matrices We usually judge a matrix by its eigenvalues, but sometimes the eigenvalues are only the bait. The real monster is hiding in the pseudospectrum. Most of the time, a matrix is shown as a few points in the complex plane called eigenvalues. But for some matrices, especially non-normal matrices, those few points do not tell the whole story. In this scene, the complex plane becomes a moving landscape. The bright anchors are the true eigenvalues. Around them, huge pseudospectral continents grow, split, and collapse as the matrix changes. The moving threads show where the matrix is most sensitive. In those regions, even a tiny change can create a huge response. This is the hidden instability of a matrix made visible. Algebra becomes a pressure map and the pseudospectrum becomes terrain. #Mathematics #LinearAlgebra #SpectralTheory #NumericalAnalysis #MatrixTheory #MathArt #MathematicalArt #STEM #Art

Pseudospectral Mirage of Matrices We usually judge a matrix by its eigenvalues, but sometimes the eigenvalues are only the bait. The real monster is hiding in the pseudospectrum. Most of the time, a matrix is shown as a few points in the complex plane called eigenvalues. But for some matrices, especially non-normal matrices, those few points do not tell the whole story. In this scene, the complex plane becomes a moving landscape. The bright anchors are the true eigenvalues. Around them, huge pseudospectral continents grow, split, and collapse as the matrix changes. The moving threads show where the matrix is most sensitive. In those regions, even a tiny change can create a huge response. This is the hidden instability of a matrix made visible. Algebra becomes a pressure map and the pseudospectrum becomes terrain. #Mathematics #LinearAlgebra #SpectralTheory #NumericalAnalysis #MatrixTheory #MathArt #MathematicalArt #STEM #Art

Mathelirium

29,171 次观看 • 1 个月前

The single most undervalued fact of linear algebra: matrices are graphs, and graphs are matrices. Encoding matrices as graphs is a cheat code, making complex behavior simple to study. Let me show you how!

The single most undervalued fact of linear algebra: matrices are graphs, and graphs are matrices. Encoding matrices as graphs is a cheat code, making complex behavior simple to study. Let me show you how!

Tivadar Danka

158,746 次观看 • 6 个月前

SVM by hand ✍️ ~ 19 steps walkthrough below (Linear vs RBF) Support Vector Machines reigned supreme in machine learning before the deep learning revolution. An SVM predicts with dot products, the same matrix multiplication every model uses. What it does not do is train by backpropagation: it is fitted by convex optimization, so there is no matrix-multiplication backward pass for a GPU to accelerate. I drew and calculated two SVMs by hand: a linear one (top) and an RBF one (bottom), classifying the same two test vectors. Goal: turn six training vectors and their learned coefficients into a prediction, and see what changing the kernel actually changes. = 1. Given = Six training vectors, their labels, and the coefficients and bias already learned. A coefficient of zero means that vector is not a support vector: too far from the boundary to matter. = 2. Linear kernel, test vector 1 = Let us take the dot product of the test vector with every training vector. The dot product stands in for cosine similarity, and the column of results is the first column of the kernel matrix K. = 3. Linear kernel, test vector 2 = We do the same for the second, and K is complete. = 4. Signed weights = Let us multiply each coefficient by its label. The second training vector drops out here, because its coefficient is 0. = 5. Weighted combination = We multiply the signed weights through K and add the bias b. The result is a signed distance to the decision boundary: 17 and 5. = 6. Classify = Let us take the sign. Both are positive. = 7 to 11. RBF kernel, test vector 1 = Now the same picture with a different kernel, in five moves: square the differences, sum them, take the square root for the L2 distance, multiply by minus gamma, and raise e to that power. The negation is what turns a distance into a similarity, and gamma controls how far a single training vector's influence reaches. = 12 to 16. RBF kernel, test vector 2 = We repeat all five. The numbers change, the moves do not. = 17 to 19. Decision boundary, again = Signed weights, weighted combination, sign. Identical arithmetic to steps 4 through 6, on a K that was built a completely different way. The outputs: Linear K, first column = [13, 25, 12, 15, 19, 27] Linear decision values = 17 and 5, both positive RBF decision values = -2 and 1, so negative and positive The takeaway: the kernel is the only thing that changed, and it changed the answer. The linear SVM calls both test vectors positive; the RBF one splits them. Everything after the kernel matrix, the signed weights and the weighted combination and the sign, is the same page of arithmetic twice. 💾 Save this post!

SVM by hand ✍️ ~ 19 steps walkthrough below (Linear vs RBF) Support Vector Machines reigned supreme in machine learning before the deep learning revolution. An SVM predicts with dot products, the same matrix multiplication every model uses. What it does not do is train by backpropagation: it is fitted by convex optimization, so there is no matrix-multiplication backward pass for a GPU to accelerate. I drew and calculated two SVMs by hand: a linear one (top) and an RBF one (bottom), classifying the same two test vectors. Goal: turn six training vectors and their learned coefficients into a prediction, and see what changing the kernel actually changes. = 1. Given = Six training vectors, their labels, and the coefficients and bias already learned. A coefficient of zero means that vector is not a support vector: too far from the boundary to matter. = 2. Linear kernel, test vector 1 = Let us take the dot product of the test vector with every training vector. The dot product stands in for cosine similarity, and the column of results is the first column of the kernel matrix K. = 3. Linear kernel, test vector 2 = We do the same for the second, and K is complete. = 4. Signed weights = Let us multiply each coefficient by its label. The second training vector drops out here, because its coefficient is 0. = 5. Weighted combination = We multiply the signed weights through K and add the bias b. The result is a signed distance to the decision boundary: 17 and 5. = 6. Classify = Let us take the sign. Both are positive. = 7 to 11. RBF kernel, test vector 1 = Now the same picture with a different kernel, in five moves: square the differences, sum them, take the square root for the L2 distance, multiply by minus gamma, and raise e to that power. The negation is what turns a distance into a similarity, and gamma controls how far a single training vector's influence reaches. = 12 to 16. RBF kernel, test vector 2 = We repeat all five. The numbers change, the moves do not. = 17 to 19. Decision boundary, again = Signed weights, weighted combination, sign. Identical arithmetic to steps 4 through 6, on a K that was built a completely different way. The outputs: Linear K, first column = [13, 25, 12, 15, 19, 27] Linear decision values = 17 and 5, both positive RBF decision values = -2 and 1, so negative and positive The takeaway: the kernel is the only thing that changed, and it changed the answer. The linear SVM calls both test vectors positive; the RBF one splits them. Everything after the kernel matrix, the signed weights and the weighted combination and the sign, is the same page of arithmetic twice. 💾 Save this post!

Tom Yeh

16,510 次观看 • 7 天前

This is a great lecture at MIT by David Shirokoff on Markov Chains. He covers the fundamentals of Markov Chains using a simple particle movement example. He starts by explaining how a particle moves between two positions, A & B, with different probabilities. From there, the talk converts the problem into matrix form using a Markov matrix. The main topics covered are: - Transition probabilities - Markov matrices - Probability vectors - Matrix multiplication in Markov Chains - Finding probabilities after n steps - Eigenvalues and eigenvectors - Matrix diagonalization - Long-term steady state distribution

This is a great lecture at MIT by David Shirokoff on Markov Chains. He covers the fundamentals of Markov Chains using a simple particle movement example. He starts by explaining how a particle moves between two positions, A & B, with different probabilities. From there, the talk converts the problem into matrix form using a Markov matrix. The main topics covered are: - Transition probabilities - Markov matrices - Probability vectors - Matrix multiplication in Markov Chains - Finding probabilities after n steps - Eigenvalues and eigenvectors - Matrix diagonalization - Long-term steady state distribution

𝗿𝗮𝗺𝗮𝗸𝗿𝘂𝘀𝗵𝗻𝗮— 𝗲/𝗮𝗰𝗰

141,246 次观看 • 2 个月前

It is possible to recreate how lighting works just by running the dot product between the normal and a vector that represents where the light is coming from #b3d #geometrynodes

It is possible to recreate how lighting works just by running the dot product between the normal and a vector that represents where the light is coming from #b3d #geometrynodes

Aumission

44,151 次观看 • 5 个月前

Introducing Operation Matrix Because Matrix operations generally break kids’ brains on whiteboards… Matrices aren't just numbers- they are transformations in space Now can actually SEE the geometry: every addition, multiplication, inverse, eigenvalue, and eigenvector becomes a transformation in real-time space. What it visualizes: • Basic ops (add/subtract/multiply) • Transpose, Invert, Adjugated • Cross products + Determinant • Matrix powers + Eigenvalues/Eigenvectors #LinearAlgebra #CreativeCoding #ThreeJS #ReactThreeFiber #MathVisualization #EdTech #WebGL #STEM React-three-fiber Three.js

Introducing Operation Matrix Because Matrix operations generally break kids’ brains on whiteboards… Matrices aren't just numbers- they are transformations in space Now can actually SEE the geometry: every addition, multiplication, inverse, eigenvalue, and eigenvector becomes a transformation in real-time space. What it visualizes: • Basic ops (add/subtract/multiply) • Transpose, Invert, Adjugated • Cross products + Determinant • Matrix powers + Eigenvalues/Eigenvectors #LinearAlgebra #CreativeCoding #ThreeJS #ReactThreeFiber #MathVisualization #EdTech #WebGL #STEM React-three-fiber Three.js

SHAHNAB AHMED

58,784 次观看 • 4 个月前

Introducing Initiatives. Coordinate strategic product efforts and monitor their progress at scale.

Introducing Initiatives. Coordinate strategic product efforts and monitor their progress at scale.

Linear

1,533,154 次观看 • 2 年前

and Iran is the first product

and Iran is the first product

Abier

33,951 次观看 • 4 个月前

Functions are vectors! This perspective lets us apply the tools of linear algebra to computational problems from image and geometry processing to machine learning and light transport—and provides a natural explanation for Fourier series. Let's explore:

Functions are vectors! This perspective lets us apply the tools of linear algebra to computational problems from image and geometry processing to machine learning and light transport—and provides a natural explanation for Fourier series. Let's explore:

Max Slater

30,424 次观看 • 3 年前

[CLIP] by Hand ✍️ The CLIP (Contrastive Language–Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: 🟨 Learn a shared embedding space for text and image [1] Given ↳ A mini batch of 3 text-image pairs ↳ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] 🟪 Text → 2 Vectors (3D) ↳ Look up word embedding vectors using word2vec. [3] 🟩 Image → 2 Vectors (4D) ↳ Divide the image into two patches. ↳ Flatten each patch [4] Process other pairs ↳ Repeat [2]-[3] [5] 🟪 Text Encoder & 🟩 Image Encoder ↳ Encode input vectors into feature vectors ↳ Here, both encoders are simple one layer perceptron (linear + ReLU) ↳ In practice, the encoders are usually transformer models. [6] 🟪 🟩 Mean Pooling: 2 → 1 vector ↳ Average 2 feature vectors into a single vector by averaging across the columns ↳ The goal is to have one vector to represent each image or text [7] 🟪 🟩 -> 🟨 Projection ↳ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). ↳ Use a linear layer to project image and text vectors to a 2D shared embedding space. 🏋️ Contrastive Pre-training 🏋️ [8] Prepare for MatMul ↳ Copy text vectors (T1,T2,T3) ↳ Copy the transpose of image vectors (I1,I2,I3) ↳ They are all in the 2D shared embedding space. [9] 🟦 MatMul ↳ Multiply T and I matrices. ↳ This is equivalent to taking dot product between every pair of image and text vectors. ↳ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] 🟦 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [11] 🟦 Softmax: ∑ ↳ Sum each row for 🟩 image→🟪 text ↳ Sum each column for 🟪 text→ 🟩 image [12] 🟦 Softmax: 1 / sum ↳ Divide each element by the column sum to obtain a similarity matrix for 🟪 text→🟩 image ↳ Divide each element by the row sum to obtain a similarity matrix for 🟩 image→🟪 text [13] 🟥 Loss Gradients ↳ The "Targets" for the similarity matrices are Identity Matrices. ↳ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. ↳ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. ↳ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. ↳ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

[CLIP] by Hand ✍️ The CLIP (Contrastive Language–Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: 🟨 Learn a shared embedding space for text and image [1] Given ↳ A mini batch of 3 text-image pairs ↳ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] 🟪 Text → 2 Vectors (3D) ↳ Look up word embedding vectors using word2vec. [3] 🟩 Image → 2 Vectors (4D) ↳ Divide the image into two patches. ↳ Flatten each patch [4] Process other pairs ↳ Repeat [2]-[3] [5] 🟪 Text Encoder & 🟩 Image Encoder ↳ Encode input vectors into feature vectors ↳ Here, both encoders are simple one layer perceptron (linear + ReLU) ↳ In practice, the encoders are usually transformer models. [6] 🟪 🟩 Mean Pooling: 2 → 1 vector ↳ Average 2 feature vectors into a single vector by averaging across the columns ↳ The goal is to have one vector to represent each image or text [7] 🟪 🟩 -> 🟨 Projection ↳ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). ↳ Use a linear layer to project image and text vectors to a 2D shared embedding space. 🏋️ Contrastive Pre-training 🏋️ [8] Prepare for MatMul ↳ Copy text vectors (T1,T2,T3) ↳ Copy the transpose of image vectors (I1,I2,I3) ↳ They are all in the 2D shared embedding space. [9] 🟦 MatMul ↳ Multiply T and I matrices. ↳ This is equivalent to taking dot product between every pair of image and text vectors. ↳ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] 🟦 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [11] 🟦 Softmax: ∑ ↳ Sum each row for 🟩 image→🟪 text ↳ Sum each column for 🟪 text→ 🟩 image [12] 🟦 Softmax: 1 / sum ↳ Divide each element by the column sum to obtain a similarity matrix for 🟪 text→🟩 image ↳ Divide each element by the row sum to obtain a similarity matrix for 🟩 image→🟪 text [13] 🟥 Loss Gradients ↳ The "Targets" for the similarity matrices are Identity Matrices. ↳ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. ↳ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. ↳ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. ↳ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

Tom Yeh

67,834 次观看 • 2 年前

Introducing Product Intelligence. AI-assisted product operations for routine, manual tasks. • Automates the overhead of triage intake • Detects duplicates and related work • Suggests the right assignee, team, labels, and more Now available in Technology Preview.

Introducing Product Intelligence. AI-assisted product operations for routine, manual tasks. • Automates the overhead of triage intake • Detects duplicates and related work • Suggests the right assignee, team, labels, and more Now available in Technology Preview.

Linear

190,481 次观看 • 11 个月前

Introducing Alloy interactive capture. > Open your product in the browser and start capturing > Click through pages or flows to create an interactive prototype > Prompt your product idea and instantly share a link Available today for all Alloy users.

Introducing Alloy interactive capture. > Open your product in the browser and start capturing > Click through pages or flows to create an interactive prototype > Prompt your product idea and instantly share a link Available today for all Alloy users.

Simon Kubica

12,046 次观看 • 6 个月前

Linear: puts the actual product on the homepage and lets you click through it. More companies should be doing this.

Linear: puts the actual product on the homepage and lets you click through it. More companies should be doing this.

Brett

132,601 次观看 • 5 个月前

Vector Database by hand ✍️ ~ 10 steps walkthrough below Vector databases are the backbone of Retrieval Augmented Generation (RAG). How do they actually work? Goal: index three sentences, then answer a query by finding the nearest one, filling in every cell yourself. = 1. Given = A dataset of three sentences, three words each. In practice it is millions of them. = 2. Word embeddings = Let us look up each word in an embedding table. Here the vocabulary is 22 words; in practice it is tens of thousands, and the vectors have thousands of dimensions rather than four. = 3. Encoding = We feed the sequence to an encoder, one linear layer and a ReLU, and get one feature vector per word. In practice the encoder is a transformer. = 4. Mean pooling = Let us average across the columns. Three word vectors collapse into one, which is what people mean by a text embedding or a sentence embedding. = 5. Indexing = We multiply by a projection matrix and the four dimensions become two. It is doing the job of a hash: a short representation that is faster to compare, and it is what gets saved in the vector storage. = 6. Process "who are you" = Let us repeat steps 2 to 5 on the second sentence. = 7. Process "who am I" = We do it a third time. The database is now indexed. = 8. Query "am I you" = Let us push the query through the very same pipeline: lookup, encoder, mean pooling, projection, and it lands as a 2D vector in the same space. = 9. Dot products = We transpose the query and multiply, which takes the dot product against every stored vector at once. The dot product is the estimate of similarity. = 10. Nearest neighbour = Let us scan for the largest: 60/9 beats 44/9 and 40/9, so the answer is "who am I". Scanning billions of vectors one at a time is what makes this the slow step in practice, which is why real databases use an approximate nearest neighbour index like HNSW. The outputs: Stored index vectors = [5/3, 2/3], [5/3, 0], [7/3, 2/3] Query vector = [8/3, 2/3] Dot products = 44/9, 40/9, 60/9 Nearest neighbour = "who am I" The takeaway: a vector database is an embedding pipeline, a projection, and a dot product. Every step here is arithmetic you can do in pen, which is worth remembering when the word "database" makes it sound like something else. 💾 Save this post!

Vector Database by hand ✍️ ~ 10 steps walkthrough below Vector databases are the backbone of Retrieval Augmented Generation (RAG). How do they actually work? Goal: index three sentences, then answer a query by finding the nearest one, filling in every cell yourself. = 1. Given = A dataset of three sentences, three words each. In practice it is millions of them. = 2. Word embeddings = Let us look up each word in an embedding table. Here the vocabulary is 22 words; in practice it is tens of thousands, and the vectors have thousands of dimensions rather than four. = 3. Encoding = We feed the sequence to an encoder, one linear layer and a ReLU, and get one feature vector per word. In practice the encoder is a transformer. = 4. Mean pooling = Let us average across the columns. Three word vectors collapse into one, which is what people mean by a text embedding or a sentence embedding. = 5. Indexing = We multiply by a projection matrix and the four dimensions become two. It is doing the job of a hash: a short representation that is faster to compare, and it is what gets saved in the vector storage. = 6. Process "who are you" = Let us repeat steps 2 to 5 on the second sentence. = 7. Process "who am I" = We do it a third time. The database is now indexed. = 8. Query "am I you" = Let us push the query through the very same pipeline: lookup, encoder, mean pooling, projection, and it lands as a 2D vector in the same space. = 9. Dot products = We transpose the query and multiply, which takes the dot product against every stored vector at once. The dot product is the estimate of similarity. = 10. Nearest neighbour = Let us scan for the largest: 60/9 beats 44/9 and 40/9, so the answer is "who am I". Scanning billions of vectors one at a time is what makes this the slow step in practice, which is why real databases use an approximate nearest neighbour index like HNSW. The outputs: Stored index vectors = [5/3, 2/3], [5/3, 0], [7/3, 2/3] Query vector = [8/3, 2/3] Dot products = 44/9, 40/9, 60/9 Nearest neighbour = "who am I" The takeaway: a vector database is an embedding pipeline, a projection, and a dot product. Every step here is arithmetic you can do in pen, which is worth remembering when the word "database" makes it sound like something else. 💾 Save this post!

Tom Yeh

35,070 次观看 • 6 天前

Introducing Refix - the world's first revenue obsessed AI Product Manager It continuously watches product, marketing, sales, support metrics; finds revenue opportunities proactively, and ships autonomously For the first time, your product can improve itself.

Introducing Refix - the world's first revenue obsessed AI Product Manager It continuously watches product, marketing, sales, support metrics; finds revenue opportunities proactively, and ships autonomously For the first time, your product can improve itself.

Neil Agarwal

50,135 次观看 • 2 个月前

Introducing Index – the new planning tool for Product Managers on Linear. ➡️ Product leaders have long deserved a modern experience for product planning and discovery. Something powerful, flexible, and integrated with the latest tools. That’s why we built Index (Index), the new standard for Product Management: 1. Every layout at your fingertips Breeze through planning with a whiteboard, table, roadmap, or board available every step of the way. One place to go, infinite flexibility, and context on your projects carried throughout. 2. Built for B2B Product Management Natively connect to your customer context, leveraging their requests, insights, and feedback all in one place. Index connects to Hubspot, Intercom, Dovetail, Attio, Gong, and more – so you can back up your vision with customer evidence. 3. The product-native companion to Linear To build a true companion to Linear for PMs, we knew we had to raise the bar. Interactions are instant, every layout is collaborative, and it fits Linear like a glove with native sync. We're live today, and you can pre-order now at

Introducing Index – the new planning tool for Product Managers on Linear. ➡️ Product leaders have long deserved a modern experience for product planning and discovery. Something powerful, flexible, and integrated with the latest tools. That’s why we built Index (Index), the new standard for Product Management: 1. Every layout at your fingertips Breeze through planning with a whiteboard, table, roadmap, or board available every step of the way. One place to go, infinite flexibility, and context on your projects carried throughout. 2. Built for B2B Product Management Natively connect to your customer context, leveraging their requests, insights, and feedback all in one place. Index connects to Hubspot, Intercom, Dovetail, Attio, Gong, and more – so you can back up your vision with customer evidence. 3. The product-native companion to Linear To build a true companion to Linear for PMs, we knew we had to raise the bar. Interactions are instant, every layout is collaborative, and it fits Linear like a glove with native sync. We're live today, and you can pre-order now at

Simon Kubica

66,073 次观看 • 1 年前