Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

I built MatmulFlow ( — an interactive tool that makes matrix multiplication dimensions visual, part of my AI by Hand ✍️ series. Matrix multiplication dimensions are confusing. Which is the inner dimension? Columns of the first or rows of the second? And when you chain five multiplications together, it...

ProfTomYeh's profile picture

Tom Yeh

55,912 subscribers

26,269 görüntüleme • 3 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning
0:12

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning

Tom Yeh

23,948 görüntüleme • 6 gün önce

[Discrete Fourier Transform] by Hand ✍️ In signal processing, the Discrete Fourier Transform (DFT) is no doubt the most important method. But the math involved is extremely complex, literally, involving a summation over a complex number term e^(-iwt). I developed this exercise to demonstrate that underneath such complexity, DFT is just a series of matrix multiplications you can calculate by hand. ✍️ Once you see that, it should not surprise you that a deep neural network, which is also a series of matrix multiplications, with activation functions in-between, can learn to perform DFT to process and analyze signals so effectively. How does DFT work? [1] Given ↳ Signals A, B, and C in the 🟧 frequency domain: ◦ A = cos(w) + 2cos(2w) ◦ B = cos(w) + cos(3w) + cos(4w) ◦ C = -cos(2w) + cos(3w) ◦ Each signal is a weighed sum of four cosine waves at frequencies 1w, 2w, 3w, and 4w. ◦ We will apply Inverse DFT to convert the signals to time domain representations, and then demonstrate DFT can convert back to their original frequency domain representations. ↳ Signal X in the 🟩 time domain. X is sampled at 10 time points 1t, 2t, …, 10t: ◦ X = [-2.5, -1.8, 3, -0.7, -1.0, -0.7, 3, -1.8, -2.5, 5] ◦ Suppose X is also a weighted sum of the same four cosine waves, but we don’t already know their weights. We will apply DFT to discover them. [2] 🟧 Frequency Matrix (F) ↳ Write the coefficients of A, B, C as a matrix F. Each signal is a row. Each frequency is a column. ↳ A → [1, 2, 0, 0] ↳ B → [1, 0, 1, 1] ↳ C → [0, 1-, 1, 0] [3] Cosine → Discrete ↳ Sample from the continuous cosine waves at discrete time points 1t, 2t, 3t, to 10t. [4] Cosine Matrix (W) ↳ Write the samples as a matrix, Each frequency is a row. Each time point is a column. [5] Inverse DFT: 🟧 Frequency → 🟩 Time ↳ Multiply the frequency matrix F and the cosine matrix W. ↳ The meaning of this multiplication is to linearly combine the four cosine waves (rows in W) into time-domain signals (rows in T) using the weights specified in F. ↳ The result is matrix T, which are signals A, B, C converted to the time domain. Each signal is a row. Each time point is a column. [6] Transpose ↳ Transpose T, converting each signal’s time domain representation from a row to a column. [7] DFT: 🟩 Time → 🟧 Frequency ↳ Multiply the cosine matrix W with the transpose of matrix T. ↳ The purpose of this multiplication is to take a dot-product between each time-domain signal (columns in the transpose of T) and each cosine wave (rows in W), which has the effect of projecting the signal onto a cosine wave to determine how much they are correlated. Zero means not correlated at all. ↳ The result is an intermediate version of the “recovered” frequency matrix where each column corresponds to a signal and each row corresponds to a frequency. ↳ Compared to the original frequency matrix F, this intermediate matrix has non-zero weights in the correct places, but scaled up by a factor of 5 (n/2, n=10). For example, signal A, originally [1,2,0,0], is recovered at [5,10,0,0]. [8] Scale ↳ Multiply each value by 2/n = 1/5 to scale down the intermediate matrix to match the magnitude of the original frequency matrix F. [9] Transpose ↳ Transpose the recovered frequency matrix back to the same orientation of the original frequency matrix F. ↳ Like magic 🪄, the result is identical to the original F, which means DFT successfully recovered the frequency components of signals A, B, C. [10] Apply DFT to X: 🟩 Time → 🟧 Frequency ↳ Now that we have some confidence in DFT’s ability to recover frequency components, we apply DFT to X’s time-domain representation by multiplying W with X. ↳ The result is the an intermediate matrix. [11] Scale ↳ Similarly, we scale down by a factor of 5 to obtain the recovered frequency components of X (a column). [12] Transpose ↳ Similarly, we transpose the recovered column to row to match the orientation of the frequency matrix. ↳ Using the coefficients [0,0,3,2], we can write the equation of X as 3cos(3w) + 2cos(4w). Notes: I hope this by hand exercise helps you understand the essence of DFT. But there is more technical details, such as: • Sine: The complete DFT math also includes sine waves that follow a similar calculation process. • Phase: Here, we assume all the cosine waves are aligned at the origin, namely, phase is 0. If a phase p is added, for example, cos(w+p), we will need to calculate the sine component and use their ratio to figure out what p is. • Magnitude: If phase is not zero, the magnitude will need to be calculated by combining both cosine and sine terms.
0:12

[Discrete Fourier Transform] by Hand ✍️ In signal processing, the Discrete Fourier Transform (DFT) is no doubt the most important method. But the math involved is extremely complex, literally, involving a summation over a complex number term e^(-iwt). I developed this exercise to demonstrate that underneath such complexity, DFT is just a series of matrix multiplications you can calculate by hand. ✍️ Once you see that, it should not surprise you that a deep neural network, which is also a series of matrix multiplications, with activation functions in-between, can learn to perform DFT to process and analyze signals so effectively. How does DFT work? [1] Given ↳ Signals A, B, and C in the 🟧 frequency domain: ◦ A = cos(w) + 2cos(2w) ◦ B = cos(w) + cos(3w) + cos(4w) ◦ C = -cos(2w) + cos(3w) ◦ Each signal is a weighed sum of four cosine waves at frequencies 1w, 2w, 3w, and 4w. ◦ We will apply Inverse DFT to convert the signals to time domain representations, and then demonstrate DFT can convert back to their original frequency domain representations. ↳ Signal X in the 🟩 time domain. X is sampled at 10 time points 1t, 2t, …, 10t: ◦ X = [-2.5, -1.8, 3, -0.7, -1.0, -0.7, 3, -1.8, -2.5, 5] ◦ Suppose X is also a weighted sum of the same four cosine waves, but we don’t already know their weights. We will apply DFT to discover them. [2] 🟧 Frequency Matrix (F) ↳ Write the coefficients of A, B, C as a matrix F. Each signal is a row. Each frequency is a column. ↳ A → [1, 2, 0, 0] ↳ B → [1, 0, 1, 1] ↳ C → [0, 1-, 1, 0] [3] Cosine → Discrete ↳ Sample from the continuous cosine waves at discrete time points 1t, 2t, 3t, to 10t. [4] Cosine Matrix (W) ↳ Write the samples as a matrix, Each frequency is a row. Each time point is a column. [5] Inverse DFT: 🟧 Frequency → 🟩 Time ↳ Multiply the frequency matrix F and the cosine matrix W. ↳ The meaning of this multiplication is to linearly combine the four cosine waves (rows in W) into time-domain signals (rows in T) using the weights specified in F. ↳ The result is matrix T, which are signals A, B, C converted to the time domain. Each signal is a row. Each time point is a column. [6] Transpose ↳ Transpose T, converting each signal’s time domain representation from a row to a column. [7] DFT: 🟩 Time → 🟧 Frequency ↳ Multiply the cosine matrix W with the transpose of matrix T. ↳ The purpose of this multiplication is to take a dot-product between each time-domain signal (columns in the transpose of T) and each cosine wave (rows in W), which has the effect of projecting the signal onto a cosine wave to determine how much they are correlated. Zero means not correlated at all. ↳ The result is an intermediate version of the “recovered” frequency matrix where each column corresponds to a signal and each row corresponds to a frequency. ↳ Compared to the original frequency matrix F, this intermediate matrix has non-zero weights in the correct places, but scaled up by a factor of 5 (n/2, n=10). For example, signal A, originally [1,2,0,0], is recovered at [5,10,0,0]. [8] Scale ↳ Multiply each value by 2/n = 1/5 to scale down the intermediate matrix to match the magnitude of the original frequency matrix F. [9] Transpose ↳ Transpose the recovered frequency matrix back to the same orientation of the original frequency matrix F. ↳ Like magic 🪄, the result is identical to the original F, which means DFT successfully recovered the frequency components of signals A, B, C. [10] Apply DFT to X: 🟩 Time → 🟧 Frequency ↳ Now that we have some confidence in DFT’s ability to recover frequency components, we apply DFT to X’s time-domain representation by multiplying W with X. ↳ The result is the an intermediate matrix. [11] Scale ↳ Similarly, we scale down by a factor of 5 to obtain the recovered frequency components of X (a column). [12] Transpose ↳ Similarly, we transpose the recovered column to row to match the orientation of the frequency matrix. ↳ Using the coefficients [0,0,3,2], we can write the equation of X as 3cos(3w) + 2cos(4w). Notes: I hope this by hand exercise helps you understand the essence of DFT. But there is more technical details, such as: • Sine: The complete DFT math also includes sine waves that follow a similar calculation process. • Phase: Here, we assume all the cosine waves are aligned at the origin, namely, phase is 0. If a phase p is added, for example, cos(w+p), we will need to calculate the sine component and use their ratio to figure out what p is. • Magnitude: If phase is not zero, the magnitude will need to be calculated by combining both cosine and sine terms.

Tom Yeh

116,622 görüntüleme • 2 yıl önce

There's no need to speak. You must only - concentrate and recall all your past life. When a man thinks of the past, he becomes kinder. ~ Stalker (1979) Tarkovsky A déjà vu is usually a glitch in the Matrix. It happens when they change something. ~ The Matrix (1999) Wachowski
0:12

There's no need to speak. You must only - concentrate and recall all your past life. When a man thinks of the past, he becomes kinder. ~ Stalker (1979) Tarkovsky A déjà vu is usually a glitch in the Matrix. It happens when they change something. ~ The Matrix (1999) Wachowski

Reconsidering Cinema

68,284 görüntüleme • 3 yıl önce

It was 1999, and a seismic cultural shift was happening. You could feel it in Fight Club & The Matrix. The hope and sincerity the 80s & early 90s were falling out of fashion. A spiritual “meaning crisis” was moving through culture. And it came for men first.
0:10

It was 1999, and a seismic cultural shift was happening. You could feel it in Fight Club & The Matrix. The hope and sincerity the 80s & early 90s were falling out of fashion. A spiritual “meaning crisis” was moving through culture. And it came for men first.

Paul Anleitner

143,590 görüntüleme • 2 ay önce

When you have an angel and a demon sitting on your right and left shoulders, this is what it means... The Tree of Life is within; the emotions and urges come from the higher dimensions of the Sefirot/Chakras/Egyptian parts of the soul. When energy flows from the left side, especially through Gevurah, that raw energy—the energy of the lion—brings anger that cannot be easily controlled. In the Egyptian system, that part of the soul is called Sekhem. From the right side, or from Chokmah and Chesed, a flow of energy brings kindness and love. There are people who receive energy only from one side, without balance. A lot of what you receive depends on the influence of the stars when you are born, and that is by design. Some are here to bring evil, some good—you need to find the balance and purify your heart.
0:10

When you have an angel and a demon sitting on your right and left shoulders, this is what it means... The Tree of Life is within; the emotions and urges come from the higher dimensions of the Sefirot/Chakras/Egyptian parts of the soul. When energy flows from the left side, especially through Gevurah, that raw energy—the energy of the lion—brings anger that cannot be easily controlled. In the Egyptian system, that part of the soul is called Sekhem. From the right side, or from Chokmah and Chesed, a flow of energy brings kindness and love. There are people who receive energy only from one side, without balance. A lot of what you receive depends on the influence of the stars when you are born, and that is by design. Some are here to bring evil, some good—you need to find the balance and purify your heart.

Open Minded Approach

167,832 görüntüleme • 2 ay önce

Self Attention vs Cross Attention by hand ✍️ Resize the matrices yourself 👉 Two attention mechanisms, side by side. Both project X into queries; both compute attention via S = Kᵀ × Q and F = V × A. The only difference is the source of K and V. Self attention uses X for everything. Q, K, and V all come from projecting X. Each X token attends to every other X token. The score matrix S is square — 128 × 128. Cross attention uses X for queries and a second sequence E for keys and values. Each X token attends to every E token instead. The score matrix S is rectangular — 64 × 128. Notice what's shared and what's not: X is the same in both — same 36 × 128 input. Q and K share the 16 dimension — that's what makes the dot product Kᵀ × Q valid in either case. V dimensions are independent: self-attention uses 12, cross-attention uses 12. The choice doesn't depend on which mechanism you're using; it depends on what output dimension your downstream layer expects.
0:18

Self Attention vs Cross Attention by hand ✍️ Resize the matrices yourself 👉 Two attention mechanisms, side by side. Both project X into queries; both compute attention via S = Kᵀ × Q and F = V × A. The only difference is the source of K and V. Self attention uses X for everything. Q, K, and V all come from projecting X. Each X token attends to every other X token. The score matrix S is square — 128 × 128. Cross attention uses X for queries and a second sequence E for keys and values. Each X token attends to every E token instead. The score matrix S is rectangular — 64 × 128. Notice what's shared and what's not: X is the same in both — same 36 × 128 input. Q and K share the 16 dimension — that's what makes the dot product Kᵀ × Q valid in either case. V dimensions are independent: self-attention uses 12, cross-attention uses 12. The choice doesn't depend on which mechanism you're using; it depends on what output dimension your downstream layer expects.

Tom Yeh

61,300 görüntüleme • 2 ay önce

Elon Musk: Make sure you accept reality for what it is. “When I said take the red pill, I didn't realize that was some politically charged phrase. I was referring to The Matrix. Make sure you accept reality for what it is. And it turns out it's also accepting right wing dogma or something. I didn't realize that at the time.” TIME Person of the Year Interview, December 13, 2021
0:17

Elon Musk: Make sure you accept reality for what it is. “When I said take the red pill, I didn't realize that was some politically charged phrase. I was referring to The Matrix. Make sure you accept reality for what it is. And it turns out it's also accepting right wing dogma or something. I didn't realize that at the time.” TIME Person of the Year Interview, December 13, 2021

ELON CLIPS

57,343 görüntüleme • 9 ay önce

We are truly living in the Matrix, which comes from the Latin word mātrix, meaning "mother" or "womb." The material world is also connected to the words matrix and matter, as we all enter the material world through the wombs of our mothers. The illusion is born from the divine feminine energy. In Hinduism and Buddhism, this is known as the Maya illusion, born of Samsara or the Bhavachakra (Wheel of Life). There is no material existence; everything is an illusion created from electromagnetic waves, just as you create an entire material illusion when you dream through the electromagnetic activity of the brain. The Great Mother is known as the Shekinah, and her consort is the divine masculine energy known as Yesod. Together, they are part of the Creator. This knowledge is present in every religion.
0:16

We are truly living in the Matrix, which comes from the Latin word mātrix, meaning "mother" or "womb." The material world is also connected to the words matrix and matter, as we all enter the material world through the wombs of our mothers. The illusion is born from the divine feminine energy. In Hinduism and Buddhism, this is known as the Maya illusion, born of Samsara or the Bhavachakra (Wheel of Life). There is no material existence; everything is an illusion created from electromagnetic waves, just as you create an entire material illusion when you dream through the electromagnetic activity of the brain. The Great Mother is known as the Shekinah, and her consort is the divine masculine energy known as Yesod. Together, they are part of the Creator. This knowledge is present in every religion.

Open Minded Approach

15,917 görüntüleme • 1 ay önce

The UFO/UAP phenomenon is ancient. It has been here long before us, and it is connected to a grand hierarchy created by God. They are coming from higher dimensions, something that religions would describe as the Tree of Life. The phenomenon is part of us, and our soul is part of this grand system. We are currently in the lowest dimension, chakra, sefirot, or part of the soul, known as Malkhut in Kabbalah, Muladhara in Hinduism, and Khat in ancient Egypt. We are made of matter, but our soul is made of fire, and it is time for an upgrade. The Geophysical Event is connected to this transformation.
0:12

The UFO/UAP phenomenon is ancient. It has been here long before us, and it is connected to a grand hierarchy created by God. They are coming from higher dimensions, something that religions would describe as the Tree of Life. The phenomenon is part of us, and our soul is part of this grand system. We are currently in the lowest dimension, chakra, sefirot, or part of the soul, known as Malkhut in Kabbalah, Muladhara in Hinduism, and Khat in ancient Egypt. We are made of matter, but our soul is made of fire, and it is time for an upgrade. The Geophysical Event is connected to this transformation.

Open Minded Approach

136,661 görüntüleme • 2 ay önce

The Matrix is within. The Archons, the demons, they are within. People search for a prison in the world, but the ancient mystery schools searched for the prison within consciousness, where everything is first created on the mental planes. The chains are not made of iron; they are made of fear and illusion. To liberate yourself, you must first go within.
0:10

The Matrix is within. The Archons, the demons, they are within. People search for a prison in the world, but the ancient mystery schools searched for the prison within consciousness, where everything is first created on the mental planes. The chains are not made of iron; they are made of fear and illusion. To liberate yourself, you must first go within.

Open Minded Approach

12,558 görüntüleme • 1 ay önce

Vector Database by Hand ✍️ Vector databases are revolutionizing how we search and analyze complex data. They have become the backbone of Retrieval Augmented Generation (#RAG). How do vector databases work? [1] Given ↳ A dataset of three sentences, each has 3 words (or tokens) ↳ In practice, a dataset may contain millions or billions of sentences. The max number of tokens may be tens of thousands (e.g., 32,768 mistral-7b). Process "how are you" [2] 🟨 Word Embeddings ↳ For each word, look up corresponding word embedding vector from a table of 22 vectors, where 22 is the vocabulary size. ↳ In practice, the vocabulary size can be tens of thousands. The word embedding dimensions are in the thousands (e.g., 1024, 4096) [3] 🟩 Encoding ↳ Feed the sequence of word embeddings to an encoder to obtain a sequence of feature vectors, one per word. ↳ Here, the encoder is a simple one layer perceptron (linear layer + ReLU) ↳ In practice, the encoder is a transformer or one of its many variants. [4] 🟩 Mean Pooling ↳ Merge the sequence of feature vectors into a single vector using "mean pooling" which is to average across the columns. ↳ The result is a single vector. We often call it "text embeddings" or "sentence embeddings." ↳ Other pooling techniques are possible, such as CLS. But mean pooling is the most common. [5] 🟦 Indexing ↳ Reduce the dimensions of the text embedding vector by a projection matrix. The reduction rate is 50% (4->2). ↳ In practice, the values in this projection matrix is much more random. ↳ The purpose is similar to that of hashing, which is to obtain a short representation to allow faster comparison and retrieval. ↳ The resulting dimension-reduced index vector is saved in the vector storage. [6] Process "who are you" ↳ Repeat [2]-[5] [7] Process "who am I" ↳ Repeat [2]-[5] Now we have indexed our dataset in the vector database. [8] 🟥 Query: "am I you" ↳ Repeat [2]-[5] ↳ The result is a 2-d query vector. [9] 🟥 Dot Products ↳ Take dot product between the query vector and database vectors. They are all 2-d. ↳ The purpose is to use dot product to estimate similarity. ↳ By transposing the query vector, this step becomes a matrix multiplication. [10] 🟥 Nearest Neighbor ↳ Find the largest dot product by linear scan. ↳ The sentence with the highest dot product is "who am I" ↳ In practice, because scanning billions of vectors is slow, we use an Approximate Nearest Neighbor (ANN) algorithm like the Hierarchical Navigable Small Worlds (HNSW).
0:17

Vector Database by Hand ✍️ Vector databases are revolutionizing how we search and analyze complex data. They have become the backbone of Retrieval Augmented Generation (#RAG). How do vector databases work? [1] Given ↳ A dataset of three sentences, each has 3 words (or tokens) ↳ In practice, a dataset may contain millions or billions of sentences. The max number of tokens may be tens of thousands (e.g., 32,768 mistral-7b). Process "how are you" [2] 🟨 Word Embeddings ↳ For each word, look up corresponding word embedding vector from a table of 22 vectors, where 22 is the vocabulary size. ↳ In practice, the vocabulary size can be tens of thousands. The word embedding dimensions are in the thousands (e.g., 1024, 4096) [3] 🟩 Encoding ↳ Feed the sequence of word embeddings to an encoder to obtain a sequence of feature vectors, one per word. ↳ Here, the encoder is a simple one layer perceptron (linear layer + ReLU) ↳ In practice, the encoder is a transformer or one of its many variants. [4] 🟩 Mean Pooling ↳ Merge the sequence of feature vectors into a single vector using "mean pooling" which is to average across the columns. ↳ The result is a single vector. We often call it "text embeddings" or "sentence embeddings." ↳ Other pooling techniques are possible, such as CLS. But mean pooling is the most common. [5] 🟦 Indexing ↳ Reduce the dimensions of the text embedding vector by a projection matrix. The reduction rate is 50% (4->2). ↳ In practice, the values in this projection matrix is much more random. ↳ The purpose is similar to that of hashing, which is to obtain a short representation to allow faster comparison and retrieval. ↳ The resulting dimension-reduced index vector is saved in the vector storage. [6] Process "who are you" ↳ Repeat [2]-[5] [7] Process "who am I" ↳ Repeat [2]-[5] Now we have indexed our dataset in the vector database. [8] 🟥 Query: "am I you" ↳ Repeat [2]-[5] ↳ The result is a 2-d query vector. [9] 🟥 Dot Products ↳ Take dot product between the query vector and database vectors. They are all 2-d. ↳ The purpose is to use dot product to estimate similarity. ↳ By transposing the query vector, this step becomes a matrix multiplication. [10] 🟥 Nearest Neighbor ↳ Find the largest dot product by linear scan. ↳ The sentence with the highest dot product is "who am I" ↳ In practice, because scanning billions of vectors is slow, we use an Approximate Nearest Neighbor (ANN) algorithm like the Hierarchical Navigable Small Worlds (HNSW).

Tom Yeh

191,994 görüntüleme • 2 yıl önce

Although this is an obvious improvement, it is still unconstitutional. A license/permit is permission to engage in an activity that would otherwise be unlawful. Concealing a firearm is protected under the Second Amendment. Thus, any law infringing upon that right is facially invalid and can be disregarded as such. When faced with an unconstitutional law, you "may ignore it and engage with impunity in the exercise of the right" which the law infringes upon. See 𝘚𝘩𝘶𝘵𝘵𝘭𝘦𝘴𝘸𝘰𝘳𝘵𝘩 𝘷. 𝘊𝘪𝘵𝘺 𝘰𝘧 𝘉𝘪𝘳𝘮𝘪𝘯𝘨𝘩𝘢𝘮.
0:12

Although this is an obvious improvement, it is still unconstitutional. A license/permit is permission to engage in an activity that would otherwise be unlawful. Concealing a firearm is protected under the Second Amendment. Thus, any law infringing upon that right is facially invalid and can be disregarded as such. When faced with an unconstitutional law, you "may ignore it and engage with impunity in the exercise of the right" which the law infringes upon. See 𝘚𝘩𝘶𝘵𝘵𝘭𝘦𝘴𝘸𝘰𝘳𝘵𝘩 𝘷. 𝘊𝘪𝘵𝘺 𝘰𝘧 𝘉𝘪𝘳𝘮𝘪𝘯𝘨𝘩𝘢𝘮.

Witsit

10,517 görüntüleme • 1 yıl önce

Do you use Runway Gen-2 to make AI videos? This is for you! I built a simple tool to help "extend" Runway videos beyond the 4 second clip limit. Just choose a video, and the tool will give you the final frame -- so that you can drop this back into Runway, and generate another AI video based on it. Let me know if this is useful to you! Link is at bottom of the video and it's live to try right now.
0:11

Do you use Runway Gen-2 to make AI videos? This is for you! I built a simple tool to help "extend" Runway videos beyond the 4 second clip limit. Just choose a video, and the tool will give you the final frame -- so that you can drop this back into Runway, and generate another AI video based on it. Let me know if this is useful to you! Link is at bottom of the video and it's live to try right now.

Benjamin De Kraker

68,384 görüntüleme • 3 yıl önce

I'd say at the very beginning of retirement you wanna be everything and you're willing to be, you love traveling around the world to meet friends and see what kind of lives people live in. and soon, you get frustrated and bored to live in an lifestyle that people thought a FIREd guy would be. you thought you're going up but you're not at the ground and it is no upside or downside at all. that's the problem. someday you'll figure out your problem is exactly the result that you got out of the Matrix. you're like trueman heading out of his little island, you've got no audience and it still a long way to go. there's no meaning but willingness for life we choosed, there's no up and down, no left or right. the fate we created, the failures we accept it, all that happened beyond our little island made our stories. and in case you don't see me, enjoy everything you're going to meet and have good days.
0:16

I'd say at the very beginning of retirement you wanna be everything and you're willing to be, you love traveling around the world to meet friends and see what kind of lives people live in. and soon, you get frustrated and bored to live in an lifestyle that people thought a FIREd guy would be. you thought you're going up but you're not at the ground and it is no upside or downside at all. that's the problem. someday you'll figure out your problem is exactly the result that you got out of the Matrix. you're like trueman heading out of his little island, you've got no audience and it still a long way to go. there's no meaning but willingness for life we choosed, there's no up and down, no left or right. the fate we created, the failures we accept it, all that happened beyond our little island made our stories. and in case you don't see me, enjoy everything you're going to meet and have good days.

郭宇 guoyu.eth

387,139 görüntüleme • 1 yıl önce

I finally got so annoyed with ctrl+c randomly failing that I built a tool for it. It monitors the content of the clipboard, and highlights the active window when a change is detected. It acts as a visual confirmation that you properly copied stuff.
0:10

I finally got so annoyed with ctrl+c randomly failing that I built a tool for it. It monitors the content of the clipboard, and highlights the active window when a change is detected. It acts as a visual confirmation that you properly copied stuff.

Kevin Gosse

59,319 görüntüleme • 1 yıl önce

messy inputs 👉🏽 polished outputs built a prototype with the idea of letting users bring together images from Lummi into a canvas, quickly wireframe a concept, and then generate polished images as a photo, illustration, 3d render, whatever—with the right lighting, shadows, cohesive colors, and all that good stuff the output is still not great... but this is where I see the future of creative tools heading—kind of like how you give ChatGPT an idea for an email—all the messy bits—and it writes it for you in any tone or style you want in a clear way. now imagine that but for design or imagery
0:16

messy inputs 👉🏽 polished outputs built a prototype with the idea of letting users bring together images from Lummi into a canvas, quickly wireframe a concept, and then generate polished images as a photo, illustration, 3d render, whatever—with the right lighting, shadows, cohesive colors, and all that good stuff the output is still not great... but this is where I see the future of creative tools heading—kind of like how you give ChatGPT an idea for an email—all the messy bits—and it writes it for you in any tone or style you want in a clear way. now imagine that but for design or imagery

Pablo Stanley

12,109 görüntüleme • 1 yıl önce

Some people criticise me for calling it the Woke Right but I genuinely don't understand how they don't see it. The woke left claimed that America, one of the most racially tolerant countries in the world, was the most racist country ever. The woke right claims America, one of the best governed countries in the world, treats its citizens worse than any other. It is true that there are racists in America and it is true that America has problems when it comes to living up to its ideals on things like free speech. But this is just victimhood for clicks and it's frankly pathetic, unpatriotic and silly.
0:18

Some people criticise me for calling it the Woke Right but I genuinely don't understand how they don't see it. The woke left claimed that America, one of the most racially tolerant countries in the world, was the most racist country ever. The woke right claims America, one of the best governed countries in the world, treats its citizens worse than any other. It is true that there are racists in America and it is true that America has problems when it comes to living up to its ideals on things like free speech. But this is just victimhood for clicks and it's frankly pathetic, unpatriotic and silly.

Konstantin Kisin

694,350 görüntüleme • 1 yıl önce

I call whoever I want the n-word whenever I deem it necessary or funny. Further, I encourage everyone to do it. Do it. Do it until such a time as the entire national consensus is that stabbing someone is so much worse than cursing or slur slinging that suggesting otherwise is objectively laughable and results in the person claiming it’s the same or worse being reduced to the level of a clown for being so ridiculously idiotic in public. We will never equate words to violence. The Right did not and will not concede this ground to you mfs. If that means Power Word N must be used at the rate at which liberals breathe the accusation of “Nazi” at any and everyone they please, so be it. The slurs will continue until they lose all meaning, or your behavior improves and not a second sooner.
0:16

I call whoever I want the n-word whenever I deem it necessary or funny. Further, I encourage everyone to do it. Do it. Do it until such a time as the entire national consensus is that stabbing someone is so much worse than cursing or slur slinging that suggesting otherwise is objectively laughable and results in the person claiming it’s the same or worse being reduced to the level of a clown for being so ridiculously idiotic in public. We will never equate words to violence. The Right did not and will not concede this ground to you mfs. If that means Power Word N must be used at the rate at which liberals breathe the accusation of “Nazi” at any and everyone they please, so be it. The slurs will continue until they lose all meaning, or your behavior improves and not a second sooner.

Black-Guy™️

141,180 görüntüleme • 7 ay önce

The Sabotaging Practice of Over Supply and Sameness in the NFT Space. The current zeitgeist of the NFT space is that the same artists are doing the same kind of work five times a year, with project after project leaving a trail of disappointment and discontent among collectors and all of us watching in disbelief as huge resources are extracted from the space over work that feels like it could be left as an "artist study." I understand that you can do what you want with your money as collectors, but we are killing the whole space with this incestuous practice. No artist is that prolific to be able to do 5 collections of 100+ pieces each every year and actually deliver innovation and some kind of creative evolution. Of course, they can pretend play that the work has something new, but there is no precedent nor proof that that has ever happened in the speed that it happens in the NFT space. Again, people are free to through away their resources on whatever they want but with this way of doing things, we more and more are going to start seeing the consequences. Oh! There are consequences? Yes. Maybe unintended, but there are. Let's see. Let's start with the loss of belief in the NFT space as somewhere where emerging artists can come and find support for their experiments. Why even bother to bring experiments, innovation, and new ways to think of art on the blockchain if the same people have all the collectors hypnotized with their magical flutes? Why even try to come to a space where taking risks and challenging the status quo (the mission of art!!!) is overlooked? This makes the NFT space a social club and not a space for art. I guess it is fine, but IMO it is a recipe for disaster. New collectors stay away because the art will slowly but surely become stale and un-challenging. Why even bother to come and see what is happening here if you can't, as a collector, see new weird and up-and-coming artists? The amount of noise emitted by the same artists doing the same art over and over, drowns out any new voices. Again. A recipe for disaster. The NFT space is becoming a space of disappointment and doubt. We think that collections going to zero one after the other, over and over, is not damaging? I feel we are kidding ourselves. Disappointment piles up, and again, the people who will hurt are the emerging artists, the new blood, the ones who are willing to risk the most and, in return, put fire in this cold space of sameness. I love this space—don't get me wrong—it has changed my life, and I believe it has a ton of potential, but things need to change for it to become a beacon of light in art. But we need to support new voices. We need to support new ideas. The challenge is huge. I hope to contribute all I can to this change. I hope more and more see how exciting it is to go out and try to discover what else is out there and move this space forward. But again, I understand the leaps of faith needed, but if there is a space that is based on that, it's the NFT space...so there is hope. We will see. 📺by Boldtron
0:12

The Sabotaging Practice of Over Supply and Sameness in the NFT Space. The current zeitgeist of the NFT space is that the same artists are doing the same kind of work five times a year, with project after project leaving a trail of disappointment and discontent among collectors and all of us watching in disbelief as huge resources are extracted from the space over work that feels like it could be left as an "artist study." I understand that you can do what you want with your money as collectors, but we are killing the whole space with this incestuous practice. No artist is that prolific to be able to do 5 collections of 100+ pieces each every year and actually deliver innovation and some kind of creative evolution. Of course, they can pretend play that the work has something new, but there is no precedent nor proof that that has ever happened in the speed that it happens in the NFT space. Again, people are free to through away their resources on whatever they want but with this way of doing things, we more and more are going to start seeing the consequences. Oh! There are consequences? Yes. Maybe unintended, but there are. Let's see. Let's start with the loss of belief in the NFT space as somewhere where emerging artists can come and find support for their experiments. Why even bother to bring experiments, innovation, and new ways to think of art on the blockchain if the same people have all the collectors hypnotized with their magical flutes? Why even try to come to a space where taking risks and challenging the status quo (the mission of art!!!) is overlooked? This makes the NFT space a social club and not a space for art. I guess it is fine, but IMO it is a recipe for disaster. New collectors stay away because the art will slowly but surely become stale and un-challenging. Why even bother to come and see what is happening here if you can't, as a collector, see new weird and up-and-coming artists? The amount of noise emitted by the same artists doing the same art over and over, drowns out any new voices. Again. A recipe for disaster. The NFT space is becoming a space of disappointment and doubt. We think that collections going to zero one after the other, over and over, is not damaging? I feel we are kidding ourselves. Disappointment piles up, and again, the people who will hurt are the emerging artists, the new blood, the ones who are willing to risk the most and, in return, put fire in this cold space of sameness. I love this space—don't get me wrong—it has changed my life, and I believe it has a ton of potential, but things need to change for it to become a beacon of light in art. But we need to support new voices. We need to support new ideas. The challenge is huge. I hope to contribute all I can to this change. I hope more and more see how exciting it is to go out and try to discover what else is out there and move this space forward. But again, I understand the leaps of faith needed, but if there is a space that is based on that, it's the NFT space...so there is hope. We will see. 📺by Boldtron

alejandro cartagena

98,261 görüntüleme • 2 yıl önce

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:
0:16

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:

elvis

18,374 görüntüleme • 2 ay önce