Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

In three-dimensional space the determinant of a matrix whose columns are three vectors equals the signed volume of the parallelepiped those vectors span; changing the middle column so the base parallelogram doubles in area while height stays fixed therefore yields a volume of two cubic units.

Matheorems

8,323 subscribers

16,450 views • 9 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

1 Comments

afshinsafaee8 days ago

how did you make this gif?

Related Videos

In 2006, Netflix offered $1,000,000 to anyone who could improve their recommendation algorithm by 10%. Over 2,000 teams competed for three years. The team that won did not use more data. They used fewer dimensions. They found a basis - a small set of independent vectors that captured everything important about 100,000,000 movie ratings. The lead mathematician on the winning team: $2,800,000 a year. A machine learning engineer at Spotify building the same kind of system: $245,000 a year. This is MIT 18.06, Lecture 9 - Gilbert Strang. Free on YouTube. Most people think independence is obvious. Two vectors pointing in different directions. Then the definition. Independence means no combination of your vectors gives the zero vector - except the trivial one where all the coefficients are zero. That's it. That's the whole definition. But watch what it unlocks. Watch the moment Strang puts three vectors in a two-dimensional plane. He doesn't even tell you which three vectors. He just draws them. And immediately says: dependent. No question. No calculation. Why? Because three vectors in two-dimensional space means more columns than rows. More unknowns than equations. That always forces a free variable. A free variable always gives a non-zero solution to Ax = 0. And that non-zero solution is a combination of the columns that produces zero. Dependence. "Three vectors in the plane have to be dependent. That's the key fact." Then the basis. A basis is vectors that are independent and span the space. Not too few, not too many. Just right. The pivot columns of any matrix form a basis for the column space. Every other basis you can think of will have exactly the same number of vectors. Then the dimension. All bases for the same space have the same number of vectors. That number is the dimension. The rank of a matrix is the dimension of its column space. The number of free variables is the dimension of the null space. And rank plus null space dimension equals the total number of columns. "I don't take the dimension of A. I take the dimension of the column space of A. If you use those words right, it shows you've got the idea right." A data scientist at Netflix building recommendation engines: $230,000 a year. A quantitative researcher at Two Sigma finding independent factors in financial markets: $350,000 a year. A computer vision engineer at Apple using low-dimensional representations for face recognition: $260,000 a year. They all needed to know how many dimensions were really there. bookmark this and watch later - after this lecture every dataset you look at will feel like a matrix waiting to be reduced to its basis.

In 2006, Netflix offered $1,000,000 to anyone who could improve their recommendation algorithm by 10%. Over 2,000 teams competed for three years. The team that won did not use more data. They used fewer dimensions. They found a basis - a small set of independent vectors that captured everything important about 100,000,000 movie ratings. The lead mathematician on the winning team: $2,800,000 a year. A machine learning engineer at Spotify building the same kind of system: $245,000 a year. This is MIT 18.06, Lecture 9 - Gilbert Strang. Free on YouTube. Most people think independence is obvious. Two vectors pointing in different directions. Then the definition. Independence means no combination of your vectors gives the zero vector - except the trivial one where all the coefficients are zero. That's it. That's the whole definition. But watch what it unlocks. Watch the moment Strang puts three vectors in a two-dimensional plane. He doesn't even tell you which three vectors. He just draws them. And immediately says: dependent. No question. No calculation. Why? Because three vectors in two-dimensional space means more columns than rows. More unknowns than equations. That always forces a free variable. A free variable always gives a non-zero solution to Ax = 0. And that non-zero solution is a combination of the columns that produces zero. Dependence. "Three vectors in the plane have to be dependent. That's the key fact." Then the basis. A basis is vectors that are independent and span the space. Not too few, not too many. Just right. The pivot columns of any matrix form a basis for the column space. Every other basis you can think of will have exactly the same number of vectors. Then the dimension. All bases for the same space have the same number of vectors. That number is the dimension. The rank of a matrix is the dimension of its column space. The number of free variables is the dimension of the null space. And rank plus null space dimension equals the total number of columns. "I don't take the dimension of A. I take the dimension of the column space of A. If you use those words right, it shows you've got the idea right." A data scientist at Netflix building recommendation engines: $230,000 a year. A quantitative researcher at Two Sigma finding independent factors in financial markets: $350,000 a year. A computer vision engineer at Apple using low-dimensional representations for face recognition: $260,000 a year. They all needed to know how many dimensions were really there. bookmark this and watch later - after this lecture every dataset you look at will feel like a matrix waiting to be reduced to its basis.

Zyphor

14,720 views • 1 month ago

$The Gaussian integral evaluates to √π: ∫_{−∞}^{∞} e^{−x²} dx = √π. The shaded region under y = e^{−x²} has this exact area. Letting I denote the integral, I² equals the volume of the surface z = e^{−(x² + y²)}, a rotationally symmetric hill whose volume is π when computed in polar coordinates.$

The Gaussian integral evaluates to √π: ∫_{−∞}^{∞} e^{−x²} dx = √π. The shaded region under y = e^{−x²} has this exact area. Letting I denote the integral, I² equals the volume of the surface z = e^{−(x² + y²)}, a rotationally symmetric hill whose volume is π when computed in polar coordinates.

Matheorems

28,660 views • 18 days ago

Polarization of light: It is a property of electromagnetic waves. Electromagnetic waves are three dimensional and polarization is a measure of the spatial orientation of electric field and magnetic field vectors.

Polarization of light: It is a property of electromagnetic waves. Electromagnetic waves are three dimensional and polarization is a measure of the spatial orientation of electric field and magnetic field vectors.

Science Simplified

11,773 views • 6 months ago

The interesting thing about mixing rasters with vectors in Rive is that you can create very cool things. In this case the dome, eyes and nose of the skull are vectors. The skull and the base of the dome are rasters. With the vectors I can illuminate the edges of the eyes and nose, or animate them in a simpler way.

The interesting thing about mixing rasters with vectors in Rive is that you can create very cool things. In this case the dome, eyes and nose of the skull are vectors. The skull and the base of the dome are rasters. With the vectors I can illuminate the edges of the eyes and nose, or animate them in a simpler way.

JcToon

44,940 views • 2 years ago

ResNet by hand ✍️ ~ 10 steps walkthrough below "Deep Residual Learning for Image Recognition" (Kaiming He, CVPR 2016) is among the most cited papers in all of deep learning. Why does it matter so much? It fixed the exploding and vanishing gradients that kept deep networks from being deep, and made thousands of layers possible. How simple was the fix? An identity matrix. Goal: push three input vectors through a residual block, then through a transformer encoder block, filling in every cell yourself. = 1. Given = A mini batch of three input vectors, 3D, and the weights of the layers ahead. = 2. Linear layer = Let us multiply by the weights, add the bias, and apply ReLU so negatives become 0. Three feature vectors out. This is F(X). = 3. Concatenate = Now the trick. Stack an identity matrix beside the second layer's weights, and stack the input vectors under the features. Draw the lines between rows and columns: those are the skip connections. The identity is the residual. = 4. Linear layer + identity = We multiply the two stacked matrices. The identity carries X straight through while the weights transform it, so a single multiplication computes F(X) + X. Apply ReLU and hand it to the next block. Now watch the same trick inside a transformer, first in attention. = 5. Attention = Let us take three input vectors in 2D, compute the attention matrix, and multiply to get attention weighted vectors. = 6. Concatenate = We stack two identities this time, two residuals, which is how you get 1 + 1, and stack the input vectors with the attention weighted ones. = 7. Add = Multiply the stacked matrices. The identity adds attention to its own input, across the columns, which is how positions get combined. And again in the feed forward layer. = 8. First layer = Let us multiply by the feed forward weights and bias, then ReLU. Three feature vectors. = 9. Concatenate = Stack and link exactly as in step 3: the residual again. = 10. Second layer + identity = We multiply, apply ReLU, and pass the result to the next encoder block. This identity adds across the rows, combining features rather than positions. Takeaway: one simple "add" is what made really deep networks possible. 💾 Save this post!

ResNet by hand ✍️ ~ 10 steps walkthrough below "Deep Residual Learning for Image Recognition" (Kaiming He, CVPR 2016) is among the most cited papers in all of deep learning. Why does it matter so much? It fixed the exploding and vanishing gradients that kept deep networks from being deep, and made thousands of layers possible. How simple was the fix? An identity matrix. Goal: push three input vectors through a residual block, then through a transformer encoder block, filling in every cell yourself. = 1. Given = A mini batch of three input vectors, 3D, and the weights of the layers ahead. = 2. Linear layer = Let us multiply by the weights, add the bias, and apply ReLU so negatives become 0. Three feature vectors out. This is F(X). = 3. Concatenate = Now the trick. Stack an identity matrix beside the second layer's weights, and stack the input vectors under the features. Draw the lines between rows and columns: those are the skip connections. The identity is the residual. = 4. Linear layer + identity = We multiply the two stacked matrices. The identity carries X straight through while the weights transform it, so a single multiplication computes F(X) + X. Apply ReLU and hand it to the next block. Now watch the same trick inside a transformer, first in attention. = 5. Attention = Let us take three input vectors in 2D, compute the attention matrix, and multiply to get attention weighted vectors. = 6. Concatenate = We stack two identities this time, two residuals, which is how you get 1 + 1, and stack the input vectors with the attention weighted ones. = 7. Add = Multiply the stacked matrices. The identity adds attention to its own input, across the columns, which is how positions get combined. And again in the feed forward layer. = 8. First layer = Let us multiply by the feed forward weights and bias, then ReLU. Three feature vectors. = 9. Concatenate = Stack and link exactly as in step 3: the residual again. = 10. Second layer + identity = We multiply, apply ReLU, and pass the result to the next encoder block. This identity adds across the rows, combining features rather than positions. Takeaway: one simple "add" is what made really deep networks possible. 💾 Save this post!

Tom Yeh

18,152 views • 2 months ago

[Self-Attention] by Hand ✍️ Self-attention is what enables LLMs to understand context. How does it work? This exercise demonstrates how to calculate a 6-3 attention head by hand. Note that if we have two instances of this, we get 6-6 attention (i.e., multi-head attention, n=2). -- 𝗚𝗼𝗮𝗹 -- Transform [6D Features 🟧] to [3D Attention Weighted Features 🟦] -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- [1] Given ↳ A set of 4 feature vectors (6-D): x1,x2,x3,x4 [2] Query, Key, Value ↳ Multiply features x's with linear transformation matrices WQ, WK, and WV, to obtain query vectors (q1,q2,q3,q4), key vectors (k1,k2,k3,k4), and value vectors (v1,v2,v3,v4). ↳ "Self" refers to the fact that both queries and keys are derived from the same set of features. [3] 🟪 Prepare for MatMul ↳ Copy query vectors ↳ Copy the transpose of key vectors [4] 🟪 MatMul ↳ Multiply K^T and Q ↳ This is equivalent to taking dot product between every pair of query and key vectors. ↳ The purpose is to use dot product as an estimate of the "matching score" between every key-value pair. ↳ This estimate makes sense because dot product is the numerator of Cosine Similarity between two vectors. [5] 🟨 Scale ↳ Scale each element by the square root of dk, which is the dimension of key vectors (dk=3). ↳ The purpose is to normalize the impact of the dk on matching scores, even if we scale dk to 32, 64, or 128. ↳ To simplify hand calculation, we approximate [ □/sqrt(3) ] with [ floor(□/2) ]. [6] 🟩 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [7] 🟩 Softmax: ∑ ↳ Sum across each column [8] 🟩 Softmax: 1 / sum ↳ For each column, divide each element by the column sum ↳ The purpose is normalize each column so that the numbers sum to 1. In other words, each column is a probability distribution of attention, and we have four of them. ↳ The result is the Attention Weight Matrix (A) (yellow) [9] 🟦 MatMul ↳ Multiply the value vectors (Vs) with the Attention Weight Matrix (A) ↳ The results are the attention weighted features Zs. ↳ They are fed to the position-wise feed forward network in the next layer.

[Self-Attention] by Hand ✍️ Self-attention is what enables LLMs to understand context. How does it work? This exercise demonstrates how to calculate a 6-3 attention head by hand. Note that if we have two instances of this, we get 6-6 attention (i.e., multi-head attention, n=2). -- 𝗚𝗼𝗮𝗹 -- Transform [6D Features 🟧] to [3D Attention Weighted Features 🟦] -- 𝗪𝗮𝗹𝗸𝘁𝗵𝗿𝗼𝘂𝗴𝗵 -- [1] Given ↳ A set of 4 feature vectors (6-D): x1,x2,x3,x4 [2] Query, Key, Value ↳ Multiply features x's with linear transformation matrices WQ, WK, and WV, to obtain query vectors (q1,q2,q3,q4), key vectors (k1,k2,k3,k4), and value vectors (v1,v2,v3,v4). ↳ "Self" refers to the fact that both queries and keys are derived from the same set of features. [3] 🟪 Prepare for MatMul ↳ Copy query vectors ↳ Copy the transpose of key vectors [4] 🟪 MatMul ↳ Multiply K^T and Q ↳ This is equivalent to taking dot product between every pair of query and key vectors. ↳ The purpose is to use dot product as an estimate of the "matching score" between every key-value pair. ↳ This estimate makes sense because dot product is the numerator of Cosine Similarity between two vectors. [5] 🟨 Scale ↳ Scale each element by the square root of dk, which is the dimension of key vectors (dk=3). ↳ The purpose is to normalize the impact of the dk on matching scores, even if we scale dk to 32, 64, or 128. ↳ To simplify hand calculation, we approximate [ □/sqrt(3) ] with [ floor(□/2) ]. [6] 🟩 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [7] 🟩 Softmax: ∑ ↳ Sum across each column [8] 🟩 Softmax: 1 / sum ↳ For each column, divide each element by the column sum ↳ The purpose is normalize each column so that the numbers sum to 1. In other words, each column is a probability distribution of attention, and we have four of them. ↳ The result is the Attention Weight Matrix (A) (yellow) [9] 🟦 MatMul ↳ Multiply the value vectors (Vs) with the Attention Weight Matrix (A) ↳ The results are the attention weighted features Zs. ↳ They are fed to the position-wise feed forward network in the next layer.

Tom Yeh

101,225 views • 2 years ago

Even though the three gravitational bodies move in chaotic and unstable ways, their center of gravity stays fixed in space. Credit: Matt Henderson

Even though the three gravitational bodies move in chaotic and unstable ways, their center of gravity stays fixed in space. Credit: Matt Henderson

The Scientific Lens

74,804 views • 6 months ago

THE THREE FRONTIERS OF OPTIMUS Elon Musk: “We’re finalizing the design of Optimus version 3. It will have essentially the manual dexterity of a human, an AI mind that can navigate and comprehend reality, and it will be made in very high volume. Those are the three really hard things.” Once these are solved, Optimus becomes the biggest product of all time.

THE THREE FRONTIERS OF OPTIMUS Elon Musk: “We’re finalizing the design of Optimus version 3. It will have essentially the manual dexterity of a human, an AI mind that can navigate and comprehend reality, and it will be made in very high volume. Those are the three really hard things.” Once these are solved, Optimus becomes the biggest product of all time.

Lacey

39,644 views • 3 months ago

[CLIP] by Hand ✍️ The CLIP (Contrastive Language–Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: 🟨 Learn a shared embedding space for text and image [1] Given ↳ A mini batch of 3 text-image pairs ↳ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] 🟪 Text → 2 Vectors (3D) ↳ Look up word embedding vectors using word2vec. [3] 🟩 Image → 2 Vectors (4D) ↳ Divide the image into two patches. ↳ Flatten each patch [4] Process other pairs ↳ Repeat [2]-[3] [5] 🟪 Text Encoder & 🟩 Image Encoder ↳ Encode input vectors into feature vectors ↳ Here, both encoders are simple one layer perceptron (linear + ReLU) ↳ In practice, the encoders are usually transformer models. [6] 🟪 🟩 Mean Pooling: 2 → 1 vector ↳ Average 2 feature vectors into a single vector by averaging across the columns ↳ The goal is to have one vector to represent each image or text [7] 🟪 🟩 -> 🟨 Projection ↳ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). ↳ Use a linear layer to project image and text vectors to a 2D shared embedding space. 🏋️ Contrastive Pre-training 🏋️ [8] Prepare for MatMul ↳ Copy text vectors (T1,T2,T3) ↳ Copy the transpose of image vectors (I1,I2,I3) ↳ They are all in the 2D shared embedding space. [9] 🟦 MatMul ↳ Multiply T and I matrices. ↳ This is equivalent to taking dot product between every pair of image and text vectors. ↳ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] 🟦 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [11] 🟦 Softmax: ∑ ↳ Sum each row for 🟩 image→🟪 text ↳ Sum each column for 🟪 text→ 🟩 image [12] 🟦 Softmax: 1 / sum ↳ Divide each element by the column sum to obtain a similarity matrix for 🟪 text→🟩 image ↳ Divide each element by the row sum to obtain a similarity matrix for 🟩 image→🟪 text [13] 🟥 Loss Gradients ↳ The "Targets" for the similarity matrices are Identity Matrices. ↳ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. ↳ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. ↳ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. ↳ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

[CLIP] by Hand ✍️ The CLIP (Contrastive Language–Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: 🟨 Learn a shared embedding space for text and image [1] Given ↳ A mini batch of 3 text-image pairs ↳ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] 🟪 Text → 2 Vectors (3D) ↳ Look up word embedding vectors using word2vec. [3] 🟩 Image → 2 Vectors (4D) ↳ Divide the image into two patches. ↳ Flatten each patch [4] Process other pairs ↳ Repeat [2]-[3] [5] 🟪 Text Encoder & 🟩 Image Encoder ↳ Encode input vectors into feature vectors ↳ Here, both encoders are simple one layer perceptron (linear + ReLU) ↳ In practice, the encoders are usually transformer models. [6] 🟪 🟩 Mean Pooling: 2 → 1 vector ↳ Average 2 feature vectors into a single vector by averaging across the columns ↳ The goal is to have one vector to represent each image or text [7] 🟪 🟩 -> 🟨 Projection ↳ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). ↳ Use a linear layer to project image and text vectors to a 2D shared embedding space. 🏋️ Contrastive Pre-training 🏋️ [8] Prepare for MatMul ↳ Copy text vectors (T1,T2,T3) ↳ Copy the transpose of image vectors (I1,I2,I3) ↳ They are all in the 2D shared embedding space. [9] 🟦 MatMul ↳ Multiply T and I matrices. ↳ This is equivalent to taking dot product between every pair of image and text vectors. ↳ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] 🟦 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [11] 🟦 Softmax: ∑ ↳ Sum each row for 🟩 image→🟪 text ↳ Sum each column for 🟪 text→ 🟩 image [12] 🟦 Softmax: 1 / sum ↳ Divide each element by the column sum to obtain a similarity matrix for 🟪 text→🟩 image ↳ Divide each element by the row sum to obtain a similarity matrix for 🟩 image→🟪 text [13] 🟥 Loss Gradients ↳ The "Targets" for the similarity matrices are Identity Matrices. ↳ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. ↳ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. ↳ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. ↳ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

Tom Yeh

67,896 views • 2 years ago

WCW Nitro was wild. They'd start hour two/three of the show with pyro and music in the middle of a match.

WCW Nitro was wild. They'd start hour two/three of the show with pyro and music in the middle of a match.

Allan

217,874 views • 2 years ago

Two similar hit-wicket dismissals in the span of three days 😯 Ryan Higgins in the Blast (via Surrey Cricket)

Two similar hit-wicket dismissals in the span of three days 😯 Ryan Higgins in the Blast (via Surrey Cricket)

Cricinfo

61,743 views • 4 months ago

Punjabis in Canada are signing a petition to reduce the sentence of Harjinder Singh, an illegal immigrant whose illegal U-turn while driving in Florida caused the death of three people

Punjabis in Canada are signing a petition to reduce the sentence of Harjinder Singh, an illegal immigrant whose illegal U-turn while driving in Florida caused the death of three people

Lord Immy Kant

118,338 views • 1 year ago

Authorities in New York City are investigating new video showing three people climbing out of a Manhattan manhole in the middle of the night.

Authorities in New York City are investigating new video showing three people climbing out of a Manhattan manhole in the middle of the night.

ABC News

782,353 views • 1 month ago

This animated map shows the population change in Germany (current boundaries) over the last 150 years. Cities display wider than they really are. Every district is one flat block of data, so the round hills are just smoothing. Height is people per area, so a mountain's volume is the actual number of people in it. Blue is below half the national average of 2024. Source:

This animated map shows the population change in Germany (current boundaries) over the last 150 years. Cities display wider than they really are. Every district is one flat block of data, so the round hills are just smoothing. Height is people per area, so a mountain's volume is the actual number of people in it. Blue is below half the national average of 2024. Source:

Simon Kuestenmacher

35,137 views • 24 days ago

NEW video of the UAP Sphere Network released by US Government: I like this one because it clearly shows the difference between the three Type 1 spheres flying in a triangular/V formation… With a separate Type 2 sphere positioned next to them. In Patrick Jackson model: The three forward spheres establish the target’s position across a two-dimensional plane. The fourth sphere functions somewhat like a rangefinder or depth-calculation node, supplying distance information. Together, the four use trilateration to track an object in three-dimensional space…potentially as part of a targeting or planetary-defense system.

NEW video of the UAP Sphere Network released by US Government: I like this one because it clearly shows the difference between the three Type 1 spheres flying in a triangular/V formation… With a separate Type 2 sphere positioned next to them. In Patrick Jackson model: The three forward spheres establish the target’s position across a two-dimensional plane. The fourth sphere functions somewhat like a rangefinder or depth-calculation node, supplying distance information. Together, the four use trilateration to track an object in three-dimensional space…potentially as part of a targeting or planetary-defense system.

Jordan Crowder

36,105 views • 20 days ago