I built MatmulFlow ( — an interactive tool that... makes matrix multiplication dimensions visual, part of my AI by Hand ✍️ series. Matrix multiplication dimensions are confusing. Which is the inner dimension? Columns of the first or rows of the second? And when you chain five multiplications together, it gets worse. The idea: represent matrices as rectangles. Shift the second matrix up and to the right. The edges that must align become obvious. The result fills in the remaining space. No memorization. You can see it. It extends to chains. Stack vertically for left-multiplication. Stack horizontally for right-multiplication. Resize any matrix and watch the dimensions "flow" through the entire chain. Give it a try!show more

i'm trying to master matrix multiplication. help me out... here. column vector notation was just a mistake right? why do i have to flip it over and put it on top when multiplying? why not just write it horizontally and on top by default?show more

Singular Value Decomposition is a matrix factorization that expresses... a matrix M as UΣVᵀ, where U and V are orthogonal matrices, and Σ is diagonal. It can interpreted as the action of a rotation, a scaling and a rotation/reflection.show more

[Discrete Fourier Transform] by Hand ✍️ In signal processing,... the Discrete Fourier Transform (DFT) is no doubt the most important method. But the math involved is extremely complex, literally, involving a summation over a complex number term e^(-iwt). I developed this exercise to demonstrate that underneath such complexity, DFT is just a series of matrix multiplications you can calculate by hand. ✍️ Once you see that, it should not surprise you that a deep neural network, which is also a series of matrix multiplications, with activation functions in-between, can learn to perform DFT to process and analyze signals so effectively. How does DFT work? [1] Given ↳ Signals A, B, and C in the 🟧 frequency domain: ◦ A = cos(w) + 2cos(2w) ◦ B = cos(w) + cos(3w) + cos(4w) ◦ C = -cos(2w) + cos(3w) ◦ Each signal is a weighed sum of four cosine waves at frequencies 1w, 2w, 3w, and 4w. ◦ We will apply Inverse DFT to convert the signals to time domain representations, and then demonstrate DFT can convert back to their original frequency domain representations. ↳ Signal X in the 🟩 time domain. X is sampled at 10 time points 1t, 2t, …, 10t: ◦ X = [-2.5, -1.8, 3, -0.7, -1.0, -0.7, 3, -1.8, -2.5, 5] ◦ Suppose X is also a weighted sum of the same four cosine waves, but we don’t already know their weights. We will apply DFT to discover them. [2] 🟧 Frequency Matrix (F) ↳ Write the coefficients of A, B, C as a matrix F. Each signal is a row. Each frequency is a column. ↳ A → [1, 2, 0, 0] ↳ B → [1, 0, 1, 1] ↳ C → [0, 1-, 1, 0] [3] Cosine → Discrete ↳ Sample from the continuous cosine waves at discrete time points 1t, 2t, 3t, to 10t. [4] Cosine Matrix (W) ↳ Write the samples as a matrix, Each frequency is a row. Each time point is a column. [5] Inverse DFT: 🟧 Frequency → 🟩 Time ↳ Multiply the frequency matrix F and the cosine matrix W. ↳ The meaning of this multiplication is to linearly combine the four cosine waves (rows in W) into time-domain signals (rows in T) using the weights specified in F. ↳ The result is matrix T, which are signals A, B, C converted to the time domain. Each signal is a row. Each time point is a column. [6] Transpose ↳ Transpose T, converting each signal’s time domain representation from a row to a column. [7] DFT: 🟩 Time → 🟧 Frequency ↳ Multiply the cosine matrix W with the transpose of matrix T. ↳ The purpose of this multiplication is to take a dot-product between each time-domain signal (columns in the transpose of T) and each cosine wave (rows in W), which has the effect of projecting the signal onto a cosine wave to determine how much they are correlated. Zero means not correlated at all. ↳ The result is an intermediate version of the “recovered” frequency matrix where each column corresponds to a signal and each row corresponds to a frequency. ↳ Compared to the original frequency matrix F, this intermediate matrix has non-zero weights in the correct places, but scaled up by a factor of 5 (n/2, n=10). For example, signal A, originally [1,2,0,0], is recovered at [5,10,0,0]. [8] Scale ↳ Multiply each value by 2/n = 1/5 to scale down the intermediate matrix to match the magnitude of the original frequency matrix F. [9] Transpose ↳ Transpose the recovered frequency matrix back to the same orientation of the original frequency matrix F. ↳ Like magic 🪄, the result is identical to the original F, which means DFT successfully recovered the frequency components of signals A, B, C. [10] Apply DFT to X: 🟩 Time → 🟧 Frequency ↳ Now that we have some confidence in DFT’s ability to recover frequency components, we apply DFT to X’s time-domain representation by multiplying W with X. ↳ The result is the an intermediate matrix. [11] Scale ↳ Similarly, we scale down by a factor of 5 to obtain the recovered frequency components of X (a column). [12] Transpose ↳ Similarly, we transpose the recovered column to row to match the orientation of the frequency matrix. ↳ Using the coefficients [0,0,3,2], we can write the equation of X as 3cos(3w) + 2cos(4w). Notes: I hope this by hand exercise helps you understand the essence of DFT. But there is more technical details, such as: • Sine: The complete DFT math also includes sine waves that follow a similar calculation process. • Phase: Here, we assume all the cosine waves are aligned at the origin, namely, phase is 0. If a phase p is added, for example, cos(w+p), we will need to calculate the sine component and use their ratio to figure out what p is. • Magnitude: If phase is not zero, the magnitude will need to be calculated by combining both cosine and sine terms.show more

No one can be told what the Matrix is.... You have to see it for yourself.show more

Keanu Reeves behind the scenes of The Matrix Reloaded... (2003) - watch it to the end for the laugh 😀show more

There's no need to speak. You must only -... concentrate and recall all your past life. When a man thinks of the past, he becomes kinder. ~ Stalker (1979) Tarkovsky A déjà vu is usually a glitch in the Matrix. It happens when they change something. ~ The Matrix (1999) Wachowskishow more

It was 1999, and a seismic cultural shift was... happening. You could feel it in Fight Club & The Matrix. The hope and sincerity the 80s & early 90s were falling out of fashion. A spiritual “meaning crisis” was moving through culture. And it came for men first.show more

When you have an angel and a demon sitting... on your right and left shoulders, this is what it means... The Tree of Life is within; the emotions and urges come from the higher dimensions of the Sefirot/Chakras/Egyptian parts of the soul. When energy flows from the left side, especially through Gevurah, that raw energy—the energy of the lion—brings anger that cannot be easily controlled. In the Egyptian system, that part of the soul is called Sekhem. From the right side, or from Chokmah and Chesed, a flow of energy brings kindness and love. There are people who receive energy only from one side, without balance. A lot of what you receive depends on the influence of the stars when you are born, and that is by design. Some are here to bring evil, some good—you need to find the balance and purify your heart.show more

Self Attention vs Cross Attention by hand ✍️ Resize... the matrices yourself 👉 Two attention mechanisms, side by side. Both project X into queries; both compute attention via S = Kᵀ × Q and F = V × A. The only difference is the source of K and V. Self attention uses X for everything. Q, K, and V all come from projecting X. Each X token attends to every other X token. The score matrix S is square — 128 × 128. Cross attention uses X for queries and a second sequence E for keys and values. Each X token attends to every E token instead. The score matrix S is rectangular — 64 × 128. Notice what's shared and what's not: X is the same in both — same 36 × 128 input. Q and K share the 16 dimension — that's what makes the dot product Kᵀ × Q valid in either case. V dimensions are independent: self-attention uses 12, cross-attention uses 12. The choice doesn't depend on which mechanism you're using; it depends on what output dimension your downstream layer expects.show more

POV: How it felt when I won my first... trade and realized that escaping the matrix was possible.show more

Elon Musk: Make sure you accept reality for what... it is. “When I said take the red pill, I didn't realize that was some politically charged phrase. I was referring to The Matrix. Make sure you accept reality for what it is. And it turns out it's also accepting right wing dogma or something. I didn't realize that at the time.” TIME Person of the Year Interview, December 13, 2021show more

We are truly living in the Matrix, which comes... from the Latin word mātrix, meaning "mother" or "womb." The material world is also connected to the words matrix and matter, as we all enter the material world through the wombs of our mothers. The illusion is born from the divine feminine energy. In Hinduism and Buddhism, this is known as the Maya illusion, born of Samsara or the Bhavachakra (Wheel of Life). There is no material existence; everything is an illusion created from electromagnetic waves, just as you create an entire material illusion when you dream through the electromagnetic activity of the brain. The Great Mother is known as the Shekinah, and her consort is the divine masculine energy known as Yesod. Together, they are part of the Creator. This knowledge is present in every religion.show more

The UFO/UAP phenomenon is ancient. It has been here... long before us, and it is connected to a grand hierarchy created by God. They are coming from higher dimensions, something that religions would describe as the Tree of Life. The phenomenon is part of us, and our soul is part of this grand system. We are currently in the lowest dimension, chakra, sefirot, or part of the soul, known as Malkhut in Kabbalah, Muladhara in Hinduism, and Khat in ancient Egypt. We are made of matter, but our soul is made of fire, and it is time for an upgrade. The Geophysical Event is connected to this transformation.show more

🚀 Matrix One Social Challenge with REY A $10,000... prize pool in $MATRIX is ready and waiting - all you have to do is share your love for Matrix One! Enter to Participate: 1️⃣ Post about Matrix One on X, and tag Matrix One ██████▒▒▒▒ 60% FAIR 2️⃣ Rack up points by earning likes and retweets on your posts. 3️⃣ Score extra points when your content attracts new followers to Matrix One ██████▒▒▒▒ 60% FAIR 4️⃣ Climb the leaderboard on Rey and let the rewards roll in! 🥇 → Prize Pool 💰 $10,000 in $MATRIX is up for grabs post TGE! Secure your place on the leaderboard to claim your share of the prize pool!show more

The Matrix is within. The Archons, the demons, they... are within. People search for a prison in the world, but the ancient mystery schools searched for the prison within consciousness, where everything is first created on the mental planes. The chains are not made of iron; they are made of fear and illusion. To liberate yourself, you must first go within.show more

Can you see the Matrix for what it truly... is ? > Increase synchronization = Enhance reward output 🔗 🕶️ Optimize or be left behind.show more

Vector Database by Hand ✍️ Vector databases are revolutionizing... how we search and analyze complex data. They have become the backbone of Retrieval Augmented Generation (#RAG). How do vector databases work? [1] Given ↳ A dataset of three sentences, each has 3 words (or tokens) ↳ In practice, a dataset may contain millions or billions of sentences. The max number of tokens may be tens of thousands (e.g., 32,768 mistral-7b). Process "how are you" [2] 🟨 Word Embeddings ↳ For each word, look up corresponding word embedding vector from a table of 22 vectors, where 22 is the vocabulary size. ↳ In practice, the vocabulary size can be tens of thousands. The word embedding dimensions are in the thousands (e.g., 1024, 4096) [3] 🟩 Encoding ↳ Feed the sequence of word embeddings to an encoder to obtain a sequence of feature vectors, one per word. ↳ Here, the encoder is a simple one layer perceptron (linear layer + ReLU) ↳ In practice, the encoder is a transformer or one of its many variants. [4] 🟩 Mean Pooling ↳ Merge the sequence of feature vectors into a single vector using "mean pooling" which is to average across the columns. ↳ The result is a single vector. We often call it "text embeddings" or "sentence embeddings." ↳ Other pooling techniques are possible, such as CLS. But mean pooling is the most common. [5] 🟦 Indexing ↳ Reduce the dimensions of the text embedding vector by a projection matrix. The reduction rate is 50% (4->2). ↳ In practice, the values in this projection matrix is much more random. ↳ The purpose is similar to that of hashing, which is to obtain a short representation to allow faster comparison and retrieval. ↳ The resulting dimension-reduced index vector is saved in the vector storage. [6] Process "who are you" ↳ Repeat [2]-[5] [7] Process "who am I" ↳ Repeat [2]-[5] Now we have indexed our dataset in the vector database. [8] 🟥 Query: "am I you" ↳ Repeat [2]-[5] ↳ The result is a 2-d query vector. [9] 🟥 Dot Products ↳ Take dot product between the query vector and database vectors. They are all 2-d. ↳ The purpose is to use dot product to estimate similarity. ↳ By transposing the query vector, this step becomes a matrix multiplication. [10] 🟥 Nearest Neighbor ↳ Find the largest dot product by linear scan. ↳ The sentence with the highest dot product is "who am I" ↳ In practice, because scanning billions of vectors is slow, we use an Approximate Nearest Neighbor (ANN) algorithm like the Hierarchical Navigable Small Worlds (HNSW).show more

“Your comment about the matrix is ironic because we... watched the first matrix and the sequel yesterday.”show more

🇺🇸 Elon Musk on why most people never make... it: "If you're in the Matrix, success was never possible. The only way to achieve success is to reprogram the Matrix." Change the system or stay stuck in it.show more

I emulated the GameCube keyboard with a Pi Pico... and discovered undocumented support for it within the Enter the Matrix game.show more