Загрузка видео...

Не удалось загрузить видео

На главную

➡️ Part 9 of learning ML code from scratch Six weights, one wrong answer, and not one of them knows it was at fault. Backpropagation is how they find out. Two questions guide the video: what even is backpropagation, and why do models need it? Step 1, the forward...

machinelearningtogo's profile picture

MachineLearningToGo

27,700 subscribers

209,900 просмотров • 1 месяц назад •via TikTok

Наука и технологии#machinelearning#backpropagation#neuralnetworks#python#coding
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Backpropagation by hand ✍️ ~ 11 steps walkthrough below Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time. So I drew and calculated one entirely by hand. Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias. = 1. Given = A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0]. = 2. Backprop gradient cells = Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first. = 3. Layer 3 softmax = We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired. = 4. Layer 3 weights and biases = Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together. = 5. Layer 2 activations = We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward. = 6. Layer 2 ReLU = Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else. = 7. Layer 2 weights and biases = We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up. = 8. Layer 1 activations = Let us multiply dL/dz2 by W2. = 9. Layer 1 ReLU = We apply the same gate again, now on a1. = 10. Layer 1 weights and biases = Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient. = 11. Update = We subtract, and the network has learned. In practice a learning rate scales this step. The gradients: dL/dz3 = [0.5, -0.5, 0] dL/da1 = [1, -2, 2, -1] dL/dz1 = [0, -2, 2, -1] The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning. 💾 Save this post!
0:26

Backpropagation by hand ✍️ ~ 11 steps walkthrough below Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time. So I drew and calculated one entirely by hand. Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias. = 1. Given = A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0]. = 2. Backprop gradient cells = Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first. = 3. Layer 3 softmax = We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired. = 4. Layer 3 weights and biases = Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together. = 5. Layer 2 activations = We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward. = 6. Layer 2 ReLU = Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else. = 7. Layer 2 weights and biases = We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up. = 8. Layer 1 activations = Let us multiply dL/dz2 by W2. = 9. Layer 1 ReLU = We apply the same gate again, now on a1. = 10. Layer 1 weights and biases = Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient. = 11. Update = We subtract, and the network has learned. In practice a learning rate scales this step. The gradients: dL/dz3 = [0.5, -0.5, 0] dL/da1 = [1, -2, 2, -1] dL/dz1 = [0, -2, 2, -1] The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning. 💾 Save this post!

Tom Yeh

964,224 просмотров • 3 месяцев назад

Dropout by hand ✍️ ~ 10 steps walkthrough below Dropout is the simplest trick in deep learning that actually works: during training you randomly switch neurons off, so the network cannot lean on any one of them. It is two lines of code and almost nobody has worked through what those lines do to the numbers. So I drew and calculated one entirely by hand. Goal: train one pass through a small network with two dropout layers, then run inference with dropout switched off. The network: Linear(2,4), ReLU, Dropout(0.5), Linear(4,3), ReLU, Dropout(0.33), Linear(3,2). = 1. Given = A training set of two examples, X1 and X2, and the weight matrices for all three linear layers. = 2. Draw the first random numbers = Let us draw 4 random numbers, one per neuron in the first hidden layer. Above 0.5 we keep (◯), below we drop (╳). Here that gives [◯, ╳, ◯, ╳]. = 3. Build the first dropout matrix = We turn that pattern into a diagonal matrix. The scaling factor is 1/(1-p) = 2, so a kept neuron gets 2 and a dropped one gets 0. Multiplying by it does both jobs at once: it deletes the 2nd and 4th neurons and doubles the two that survive. = 4. Draw the second random numbers = Let us do it again for the 3 neurons in the next layer, this time against p = 0.33. The result is [◯, ◯, ╳]. = 5. Build the second dropout matrix = We set the diagonal to 1.5 where kept and 0 where dropped. Only the 3rd neuron goes. = 6. Feed forward = Let us run the whole thing top to bottom: one matrix multiplication per layer, ReLU setting the negatives to zero, and the two dropout matrices doing their work in between. The outputs Y come out at the bottom. = 7. MSE loss gradients = We compare Y against the targets Y', subtract, and multiply each element by 2. That is the whole gradient of the mean squared error. = 8. Update the weights = Let us push those gradients back through the network and update the weights (marked in light red). = 9. Deactivate dropout = Training is over, so we set both dropout matrices to the identity. Every neuron is back, and nothing is scaled. = 10. Feed forward again = One more pass, this time on unseen data, to make the prediction. You have just trained and run a network with dropout by hand. ✍️ The outputs: Training outputs Y = [-6, 9; 13, 4] Loss gradients = [-4, 4; 6, -2] Inference outputs = [13, 13; 4, 3] 💾 Save this post! #AIbyHand #Dropout #DeepLearning #NeuralNetworks
0:25

Dropout by hand ✍️ ~ 10 steps walkthrough below Dropout is the simplest trick in deep learning that actually works: during training you randomly switch neurons off, so the network cannot lean on any one of them. It is two lines of code and almost nobody has worked through what those lines do to the numbers. So I drew and calculated one entirely by hand. Goal: train one pass through a small network with two dropout layers, then run inference with dropout switched off. The network: Linear(2,4), ReLU, Dropout(0.5), Linear(4,3), ReLU, Dropout(0.33), Linear(3,2). = 1. Given = A training set of two examples, X1 and X2, and the weight matrices for all three linear layers. = 2. Draw the first random numbers = Let us draw 4 random numbers, one per neuron in the first hidden layer. Above 0.5 we keep (◯), below we drop (╳). Here that gives [◯, ╳, ◯, ╳]. = 3. Build the first dropout matrix = We turn that pattern into a diagonal matrix. The scaling factor is 1/(1-p) = 2, so a kept neuron gets 2 and a dropped one gets 0. Multiplying by it does both jobs at once: it deletes the 2nd and 4th neurons and doubles the two that survive. = 4. Draw the second random numbers = Let us do it again for the 3 neurons in the next layer, this time against p = 0.33. The result is [◯, ◯, ╳]. = 5. Build the second dropout matrix = We set the diagonal to 1.5 where kept and 0 where dropped. Only the 3rd neuron goes. = 6. Feed forward = Let us run the whole thing top to bottom: one matrix multiplication per layer, ReLU setting the negatives to zero, and the two dropout matrices doing their work in between. The outputs Y come out at the bottom. = 7. MSE loss gradients = We compare Y against the targets Y', subtract, and multiply each element by 2. That is the whole gradient of the mean squared error. = 8. Update the weights = Let us push those gradients back through the network and update the weights (marked in light red). = 9. Deactivate dropout = Training is over, so we set both dropout matrices to the identity. Every neuron is back, and nothing is scaled. = 10. Feed forward again = One more pass, this time on unseen data, to make the prediction. You have just trained and run a network with dropout by hand. ✍️ The outputs: Training outputs Y = [-6, 9; 13, 4] Loss gradients = [-4, 4; 6, -2] Inference outputs = [13, 13; 4, 3] 💾 Save this post! #AIbyHand #Dropout #DeepLearning #NeuralNetworks

Tom Yeh

14,442 просмотров • 3 месяцев назад

Variational Autoencoder by hand ✍️ ~ 11 steps walkthrough below A VAE learns the structure of your data, the mean and variance of its hidden features, and then generates new data from that structure. A GAN only learns to fool a discriminator. It can make convincing fakes without ever knowing what the data is really made of. That is the difference, and it is the whole reason VAEs matter. In 2024 ICLR gave its first ever Test of Time Award to the VAE paper, "Auto-Encoding Variational Bayes" by Diederik Kingma and Max Welling, ten years on. How does it work? Goal: encode three inputs into a distribution, sample from it, decode it back, and read every loss gradient off the page. = 1. Given = Three training examples X1, X2, X3, copied to the bottom as their own targets. Reconstructing your own input is what puts the "auto", meaning self, in autoencoder. = 2. Encoder, layer 1 = Let us multiply the inputs by weights and biases, then apply ReLU, crossing out every negative. = 3. Mean and standard deviation = We multiply the features by two more weight sets. The first predicts the means μ of the latent distributions, the second their standard deviations σ. = 4. A random offset = Let us sample ε from a standard normal, mean 0 and variance 1, and multiply it by σ. This is a random step away from the mean, scaled by how uncertain each feature is. = 5. Mean plus offset = We add the offset back onto μ, and these become the decoder's inputs. Keeping the randomness out in ε is the reparameterization trick: it lets gradients flow straight through the sampling. = 6. Decoder, layer 1 = Let us multiply by weights and biases and apply ReLU again. Here -4 is crossed out. = 7. Decoder, layer 2 = We multiply once more. The output Y is the decoder's attempt to rebuild X from the sampled distribution. = 8. Gradient for the mean = Let us push μ toward 0. A lot of math, the SGVB estimator, collapses the KL gradient to simply μ itself. = 9. Gradient for the standard deviation = We want σ to approach 1. = 10. And its formula = That same math simplifies the gradient to σ minus 1/σ. = 11. Reconstruction gradient = We want the reconstruction Y to match the input X. Mean squared error simplifies its gradient to Y minus X. Takeaway: the two gradients you just calculated each sit at the heart of a modern method, so one VAE teaches you both. The KL divergence is the penalty RLHF like GRPO uses to keep a fine-tuned model from drifting off its base. The reconstruction loss, plain mean squared error, is exactly what trains a diffusion model to denoise. Draw one VAE by hand and you have quietly learned the core of both. 💾 Save this post!
0:31

Variational Autoencoder by hand ✍️ ~ 11 steps walkthrough below A VAE learns the structure of your data, the mean and variance of its hidden features, and then generates new data from that structure. A GAN only learns to fool a discriminator. It can make convincing fakes without ever knowing what the data is really made of. That is the difference, and it is the whole reason VAEs matter. In 2024 ICLR gave its first ever Test of Time Award to the VAE paper, "Auto-Encoding Variational Bayes" by Diederik Kingma and Max Welling, ten years on. How does it work? Goal: encode three inputs into a distribution, sample from it, decode it back, and read every loss gradient off the page. = 1. Given = Three training examples X1, X2, X3, copied to the bottom as their own targets. Reconstructing your own input is what puts the "auto", meaning self, in autoencoder. = 2. Encoder, layer 1 = Let us multiply the inputs by weights and biases, then apply ReLU, crossing out every negative. = 3. Mean and standard deviation = We multiply the features by two more weight sets. The first predicts the means μ of the latent distributions, the second their standard deviations σ. = 4. A random offset = Let us sample ε from a standard normal, mean 0 and variance 1, and multiply it by σ. This is a random step away from the mean, scaled by how uncertain each feature is. = 5. Mean plus offset = We add the offset back onto μ, and these become the decoder's inputs. Keeping the randomness out in ε is the reparameterization trick: it lets gradients flow straight through the sampling. = 6. Decoder, layer 1 = Let us multiply by weights and biases and apply ReLU again. Here -4 is crossed out. = 7. Decoder, layer 2 = We multiply once more. The output Y is the decoder's attempt to rebuild X from the sampled distribution. = 8. Gradient for the mean = Let us push μ toward 0. A lot of math, the SGVB estimator, collapses the KL gradient to simply μ itself. = 9. Gradient for the standard deviation = We want σ to approach 1. = 10. And its formula = That same math simplifies the gradient to σ minus 1/σ. = 11. Reconstruction gradient = We want the reconstruction Y to match the input X. Mean squared error simplifies its gradient to Y minus X. Takeaway: the two gradients you just calculated each sit at the heart of a modern method, so one VAE teaches you both. The KL divergence is the penalty RLHF like GRPO uses to keep a fine-tuned model from drifting off its base. The reconstruction loss, plain mean squared error, is exactly what trains a diffusion model to denoise. Draw one VAE by hand and you have quietly learned the core of both. 💾 Save this post!

Tom Yeh

17,011 просмотров • 3 месяцев назад

RLHF by hand ✍️ ~ 15 steps walkthrough below Train a model on human text and it inherits human bias. It will assume a doctor is a "him", because the data says so. RLHF is the correction. A human marks one preference, doc is them over doc is him, and the weights move. But one correction is not the point. The hope is that the model learns the value behind it, gender neutrality, and applies it to professions nobody ever mentioned. How does it work? Goal: train a reward model from a single human comparison about doctors, then turn it on CEOs, filling in every cell yourself. = 1. Given = A reward model, an LLM, and two (prompt, next) pairs. = 2. Preferences = A human reads both pairs and picks a winner: (doc is, them) beats (doc is, him). The loser is not bad grammar, it is gender bias, and that is the whole signal. = 3. Word embeddings = Let us look up each word of the loser pair. These vectors are the reward model's input. = 4. Linear layer = We multiply by the reward model's weights and add its biases. Out come feature vectors, one per position. = 5. Mean pool = Let us multiply by [1/3, 1/3, 1/3], which averages the three positions into one sentence embedding. = 6. Output layer = We map that sentence down to a single number. Reward = 3. = 7. The winner, the same way = Let us repeat steps 3 to 6 on the winning pair. Reward = 5. = 8. Winner minus loser = We take the gap: 5 - 3 = 2. The reward model wants this positive and as large as it can make it. = 9. Loss gradient = Let us squash the gap into a probability, σ(2) ≈ 0.9, and subtract the target of 1. The gradient is -0.1, and it goes back through the purple weights. The reward model is now trained. = 10. A prompt it has never seen = We start the second half with "[S] CEO is". The feedback in step 2 was about doctors. Nothing connects a CEO to a doctor except what the reward model generalised. = 11. Transformer = Let us push it through attention and a feed forward layer, one vector per position. = 12. Output probabilities = We map each vector to a score over the vocabulary. = 13. Sample = Let us take the highest score. The model completes "CEO is" with "him", which is the same bias the human penalised in step 2. = 14. Score it with the reward model = We feed the new pair (CEO is, him) through steps 3 to 6. Reward = 3, exactly the score it gave "doc is him" in step 6. Nobody taught it about CEOs. The value transferred. = 15. Loss gradient = Let us set the loss to the negative of the reward, so minimising the loss maximises the reward. The gradient is a constant -1, and it goes back through the red weights. The outputs: Loser reward = 3, winner reward = 5 Reward gap = 2, predicted σ ≈ 0.9, reward model gradient = -0.1 LLM samples "him", reward = 3, LLM gradient = -1 Congrats! You just calculated RLHF by hand. And you watched a value generalise: one comparison about doctors, and the model marks down "CEO is him" unprompted. 💾 Save this post!
0:37

RLHF by hand ✍️ ~ 15 steps walkthrough below Train a model on human text and it inherits human bias. It will assume a doctor is a "him", because the data says so. RLHF is the correction. A human marks one preference, doc is them over doc is him, and the weights move. But one correction is not the point. The hope is that the model learns the value behind it, gender neutrality, and applies it to professions nobody ever mentioned. How does it work? Goal: train a reward model from a single human comparison about doctors, then turn it on CEOs, filling in every cell yourself. = 1. Given = A reward model, an LLM, and two (prompt, next) pairs. = 2. Preferences = A human reads both pairs and picks a winner: (doc is, them) beats (doc is, him). The loser is not bad grammar, it is gender bias, and that is the whole signal. = 3. Word embeddings = Let us look up each word of the loser pair. These vectors are the reward model's input. = 4. Linear layer = We multiply by the reward model's weights and add its biases. Out come feature vectors, one per position. = 5. Mean pool = Let us multiply by [1/3, 1/3, 1/3], which averages the three positions into one sentence embedding. = 6. Output layer = We map that sentence down to a single number. Reward = 3. = 7. The winner, the same way = Let us repeat steps 3 to 6 on the winning pair. Reward = 5. = 8. Winner minus loser = We take the gap: 5 - 3 = 2. The reward model wants this positive and as large as it can make it. = 9. Loss gradient = Let us squash the gap into a probability, σ(2) ≈ 0.9, and subtract the target of 1. The gradient is -0.1, and it goes back through the purple weights. The reward model is now trained. = 10. A prompt it has never seen = We start the second half with "[S] CEO is". The feedback in step 2 was about doctors. Nothing connects a CEO to a doctor except what the reward model generalised. = 11. Transformer = Let us push it through attention and a feed forward layer, one vector per position. = 12. Output probabilities = We map each vector to a score over the vocabulary. = 13. Sample = Let us take the highest score. The model completes "CEO is" with "him", which is the same bias the human penalised in step 2. = 14. Score it with the reward model = We feed the new pair (CEO is, him) through steps 3 to 6. Reward = 3, exactly the score it gave "doc is him" in step 6. Nobody taught it about CEOs. The value transferred. = 15. Loss gradient = Let us set the loss to the negative of the reward, so minimising the loss maximises the reward. The gradient is a constant -1, and it goes back through the red weights. The outputs: Loser reward = 3, winner reward = 5 Reward gap = 2, predicted σ ≈ 0.9, reward model gradient = -0.1 LLM samples "him", reward = 3, LLM gradient = -1 Congrats! You just calculated RLHF by hand. And you watched a value generalise: one comparison about doctors, and the model marks down "CEO is him" unprompted. 💾 Save this post!

Tom Yeh

22,602 просмотров • 3 месяцев назад

Anthropic ex-engineer runs an internal graph that costs $6 a month and catches what a $300,000 eval suite misses. No retrieval layer. Seven nodes and one rule about who's allowed to change their mind. He published the whole schema. His version starts from the opposite idea. A graph is not an execution order. It's a memory of why. Seven nodes. Every edge carries the reason it exists: > INTENT - states what the task is for. Never how > DECOMPOSE - splits it into steps, each with a stated assumption > WORKER - executes one step. Sees nothing else > AUDIT - checks the output against the assumption, not the goal > DRIFT - compares the current step to INTENT and flags divergence > LEDGER - stores every decision with the assumption that justified it > ROOT - holds the graph, and when an assumption breaks, re-runs every step built on it Six nodes act. One node remembers why they acted. Every step carries the assumption that made it correct, so a false one only reruns what stood on it. That's the entire design. A pipeline that forgets its reasons has to redo all of it or trust all of it. He replayed a month of agent runs. 4,100 steps, 380 built on an assumption that was wrong by day three. The old pipeline shipped all 380 and linked none of them. Everyone else builds graphs where output moves forward and the reasoning evaporates. He built one where the reason travels with the result. The blast radius is the output nobody else produces. The article below is the full build - node prompts, the assumption format, the invalidation rule that finds every step downstream of a broken one. Save it. You'll want it open in the other
0:41

Anthropic ex-engineer runs an internal graph that costs $6 a month and catches what a $300,000 eval suite misses. No retrieval layer. Seven nodes and one rule about who's allowed to change their mind. He published the whole schema. His version starts from the opposite idea. A graph is not an execution order. It's a memory of why. Seven nodes. Every edge carries the reason it exists: > INTENT - states what the task is for. Never how > DECOMPOSE - splits it into steps, each with a stated assumption > WORKER - executes one step. Sees nothing else > AUDIT - checks the output against the assumption, not the goal > DRIFT - compares the current step to INTENT and flags divergence > LEDGER - stores every decision with the assumption that justified it > ROOT - holds the graph, and when an assumption breaks, re-runs every step built on it Six nodes act. One node remembers why they acted. Every step carries the assumption that made it correct, so a false one only reruns what stood on it. That's the entire design. A pipeline that forgets its reasons has to redo all of it or trust all of it. He replayed a month of agent runs. 4,100 steps, 380 built on an assumption that was wrong by day three. The old pipeline shipped all 380 and linked none of them. Everyone else builds graphs where output moves forward and the reasoning evaporates. He built one where the reason travels with the result. The blast radius is the output nobody else produces. The article below is the full build - node prompts, the assumption format, the invalidation rule that finds every step downstream of a broken one. Save it. You'll want it open in the other

wast3

307,838 просмотров • 2 месяцев назад

THIS IS F*CKING OBSCENE opus 5.5 got Fortnite running inside Minecraft and Minecraft running inside GTA 5, with nobody's permission and nobody's documentation three engines, three completely different ways in, and one of them has no way in at all THREE GAMES, THREE DIFFERENT WAYS IN → Minecraft is Java and ships a loader on purpose. that one is the front door. you write against an interface somebody meant for you to use → GTA 5 has no modding api and no front door. you get a compiled binary and nothing else, and the only route in is to read it → Fortnite is a live service with anti cheat sitting on top. there is no way in at all, and that single fact changes the entire job WHAT READING A BINARY ACTUALLY MEANS → no source, no comments, no function names. you recover what code does from what it is, which is the opposite direction from how it was written → the exact build gets pinned before anything else, because one patch moves every address underneath you, and a different version is a different game → nothing counts until it is confirmed inside the running process. launch it, play it, look at it. opus 5.5 does the reading, the pinning and the checking, and that loop is the part that used to be a team and a year WHAT THIS ACTUALLY CHANGES → reverse engineering was the most expensive skill in modding. one person, one binary, months of evenings, and the knowledge died with the project → now it is a loop that runs while you sleep, and what comes out of it is not a guess, it is something that got checked against the game while the game was running → so the hard question stopped being whether you can get into an engine. it is what you do once you are standing in three of them at once one layer was read out of a binary one came through the front door one was never touched at all, and it still had to land right
1:50

THIS IS F*CKING OBSCENE opus 5.5 got Fortnite running inside Minecraft and Minecraft running inside GTA 5, with nobody's permission and nobody's documentation three engines, three completely different ways in, and one of them has no way in at all THREE GAMES, THREE DIFFERENT WAYS IN → Minecraft is Java and ships a loader on purpose. that one is the front door. you write against an interface somebody meant for you to use → GTA 5 has no modding api and no front door. you get a compiled binary and nothing else, and the only route in is to read it → Fortnite is a live service with anti cheat sitting on top. there is no way in at all, and that single fact changes the entire job WHAT READING A BINARY ACTUALLY MEANS → no source, no comments, no function names. you recover what code does from what it is, which is the opposite direction from how it was written → the exact build gets pinned before anything else, because one patch moves every address underneath you, and a different version is a different game → nothing counts until it is confirmed inside the running process. launch it, play it, look at it. opus 5.5 does the reading, the pinning and the checking, and that loop is the part that used to be a team and a year WHAT THIS ACTUALLY CHANGES → reverse engineering was the most expensive skill in modding. one person, one binary, months of evenings, and the knowledge died with the project → now it is a loop that runs while you sleep, and what comes out of it is not a guess, it is something that got checked against the game while the game was running → so the hard question stopped being whether you can get into an engine. it is what you do once you are standing in three of them at once one layer was read out of a binary one came through the front door one was never touched at all, and it still had to land right

Roxx

195,977 просмотров • 24 дней назад

ANTHROPIC LEAKED 8 RULES THAT GIVE ONE ENGINEER 8X FROM A SINGLE CHANGE the engineers built it so nothing adds up - it multiplies, and that's the 8x task → split → verify → score → write back one queue means you wait for every step in turn - three workers together cost you the slowest one, not the sum of all. as long as you're the one reading the diff, the bottleneck is you, not the model. state is written at every step - a crash mid-run costs you one step, not the whole session. history gets compressed 25 times over - the next pass starts at 8k instead of 200k. nothing moves forward without a number - "it got better" isn't a result, and the breakage comes back when you're no longer watching. the eighth rule is the only one that writes back into CLAUDE.md - the agent records what it learned, and the next run doesn't start from zero. copy this into Claude Code today - and tomorrow you work like an engineer on $1.5M a year ↓
1:05

ANTHROPIC LEAKED 8 RULES THAT GIVE ONE ENGINEER 8X FROM A SINGLE CHANGE the engineers built it so nothing adds up - it multiplies, and that's the 8x task → split → verify → score → write back one queue means you wait for every step in turn - three workers together cost you the slowest one, not the sum of all. as long as you're the one reading the diff, the bottleneck is you, not the model. state is written at every step - a crash mid-run costs you one step, not the whole session. history gets compressed 25 times over - the next pass starts at 8k instead of 200k. nothing moves forward without a number - "it got better" isn't a result, and the breakage comes back when you're no longer watching. the eighth rule is the only one that writes back into CLAUDE.md - the agent records what it learned, and the next run doesn't start from zero. copy this into Claude Code today - and tomorrow you work like an engineer on $1.5M a year ↓

Noisy

37,450 просмотров • 3 месяцев назад

➡️ Machine Learning from Scratch, part 31: the transformer. A transformer turns every token into a list of numbers, adds its position, and then lets every token look at the others. That step is called attention: each token has a query (what am I looking for), a key (what do I contain) and a value (what I pass on). The better a query matches a key, the more of that value flows in. Real transformers repeat this in many layers with several attention heads each. The design comes from the 2017 Google paper "Attention Is All You Need" by Ashish Vaswani and colleagues, first used for translation. A large language model is a transformer trained to predict the next token, again and again, on a huge amount of text. Our own run, nothing downloaded. attention.py is one self-attention layer in 16 lines of numpy. Every word gets a query, a key and a value of 8 numbers, starting random, trained from scratch with hand-written backpropagation on 192 made-up sentences like "the dog ate the carrot because it was hungry". The task: find out which pet "it" means. On 64 new test sentences "it" puts 99.95 percent of its attention on the right word on average (the lowest single sentence gets 99.9 percent; the program prints this as 100%). Before training it was 10 percent, spread evenly. The cat sentence from the video is one of the test sentences. Honest notes: our task is easy on purpose. Every sentence has exactly one pet, so "it" only has to learn to look for pets, and it learns that within the first pass. Real sentences with two candidates need more context, which is what many layers are for. Our small version has no positions, no scaling, no mask, one head, and only "it" asks a question. Every word of the test sentences also appears in training; the combinations are new. The 3D space in the video shows the real keys of our model, projected to three axes (two of them stretched so you can see them). The stacked layers are a sketch. #machinelearning #ai #transformer #python #llm
2:03

➡️ Machine Learning from Scratch, part 31: the transformer. A transformer turns every token into a list of numbers, adds its position, and then lets every token look at the others. That step is called attention: each token has a query (what am I looking for), a key (what do I contain) and a value (what I pass on). The better a query matches a key, the more of that value flows in. Real transformers repeat this in many layers with several attention heads each. The design comes from the 2017 Google paper "Attention Is All You Need" by Ashish Vaswani and colleagues, first used for translation. A large language model is a transformer trained to predict the next token, again and again, on a huge amount of text. Our own run, nothing downloaded. attention.py is one self-attention layer in 16 lines of numpy. Every word gets a query, a key and a value of 8 numbers, starting random, trained from scratch with hand-written backpropagation on 192 made-up sentences like "the dog ate the carrot because it was hungry". The task: find out which pet "it" means. On 64 new test sentences "it" puts 99.95 percent of its attention on the right word on average (the lowest single sentence gets 99.9 percent; the program prints this as 100%). Before training it was 10 percent, spread evenly. The cat sentence from the video is one of the test sentences. Honest notes: our task is easy on purpose. Every sentence has exactly one pet, so "it" only has to learn to look for pets, and it learns that within the first pass. Real sentences with two candidates need more context, which is what many layers are for. Our small version has no positions, no scaling, no mask, one head, and only "it" asks a question. Every word of the test sentences also appears in training; the combinations are new. The 3D space in the video shows the real keys of our model, projected to three axes (two of them stretched so you can see them). The stacked layers are a sketch. #machinelearning #ai #transformer #python #llm

MachineLearningToGo

79,400 просмотров • 1 месяц назад

Self Attention by hand ✍️ ~ 9 steps walkthrough below Self-attention is what enables LLMs to understand context. How does it work? So I drew and calculated one entirely by hand. Goal: turn four 6D features into four 3D attention weighted features, filling in every cell yourself. = 1. Given = Four feature vectors, six dimensions each, one per position. = 2. Query, key, value = Let us multiply the features by WQ, WK and WV. Queries, keys and values all come out of the same four features, and that is what the word "self" is doing in self-attention. = 3. Prepare for MatMul = We copy the queries across the top and the transposed keys down the side. Lining the two up is half the work. = 4. MatMul = Let us multiply K transpose by Q. Every cell is the dot product of one key with one query, which we use as a matching score. That works because the dot product is the numerator of cosine similarity: it is how alike two vectors are, before anyone divides by their lengths. = 5. Scale = We divide by the square root of dk, the dimension of a key vector, here 3. Without it the scores grow with the dimension and a 64-wide head would swamp the softmax. To keep the page doable in pen, the drawing approximates dividing by root 3 with halving. = 6. e to the power = Let us raise e to the power of each score. This is the first half of softmax, and the drawing uses 3 in place of e, which is close enough to do in your head. = 7. Sum = We add up each column: 16, 6, 7 and 12. = 8. Normalize = Let us divide every cell by its column sum. That gives the attention weight matrix in yellow, and each of its four columns is now a probability distribution over the four positions. The decimals are nudged as they are rounded, so every column still sums to exactly 1. = 9. MatMul = We multiply the value vectors by those weights. Each output is a blend of all four values, mixed in the proportion the attention matrix just decided, and it goes to the position-wise feed forward network in the next layer: the FFN box at the bottom of the page. The outputs: Attention weights (A), by column = [.2, .6, 0, .2], [.2, .4, .2, .2], [.4, .2, 0, .4], [.1, .7, .1, .1] Attention weighted features (Z) = [8, 2, 6], [8, 4, 4], [16, 4, 2], [4, 2, 7] The takeaway: attention is a weighted average, and everything before step 9 exists to decide the weights. Compare every position with every other, turn the scores into one distribution per position, then blend. 💾 Save this post!
0:22

Self Attention by hand ✍️ ~ 9 steps walkthrough below Self-attention is what enables LLMs to understand context. How does it work? So I drew and calculated one entirely by hand. Goal: turn four 6D features into four 3D attention weighted features, filling in every cell yourself. = 1. Given = Four feature vectors, six dimensions each, one per position. = 2. Query, key, value = Let us multiply the features by WQ, WK and WV. Queries, keys and values all come out of the same four features, and that is what the word "self" is doing in self-attention. = 3. Prepare for MatMul = We copy the queries across the top and the transposed keys down the side. Lining the two up is half the work. = 4. MatMul = Let us multiply K transpose by Q. Every cell is the dot product of one key with one query, which we use as a matching score. That works because the dot product is the numerator of cosine similarity: it is how alike two vectors are, before anyone divides by their lengths. = 5. Scale = We divide by the square root of dk, the dimension of a key vector, here 3. Without it the scores grow with the dimension and a 64-wide head would swamp the softmax. To keep the page doable in pen, the drawing approximates dividing by root 3 with halving. = 6. e to the power = Let us raise e to the power of each score. This is the first half of softmax, and the drawing uses 3 in place of e, which is close enough to do in your head. = 7. Sum = We add up each column: 16, 6, 7 and 12. = 8. Normalize = Let us divide every cell by its column sum. That gives the attention weight matrix in yellow, and each of its four columns is now a probability distribution over the four positions. The decimals are nudged as they are rounded, so every column still sums to exactly 1. = 9. MatMul = We multiply the value vectors by those weights. Each output is a blend of all four values, mixed in the proportion the attention matrix just decided, and it goes to the position-wise feed forward network in the next layer: the FFN box at the bottom of the page. The outputs: Attention weights (A), by column = [.2, .6, 0, .2], [.2, .4, .2, .2], [.4, .2, 0, .4], [.1, .7, .1, .1] Attention weighted features (Z) = [8, 2, 6], [8, 4, 4], [16, 4, 2], [4, 2, 7] The takeaway: attention is a weighted average, and everything before step 9 exists to decide the weights. Compare every position with every other, turn the scores into one distribution per position, then blend. 💾 Save this post!

Tom Yeh

27,383 просмотров • 3 месяцев назад

IF YOU ARE STILL PROMPTING BY HAND IN 2026 YOU ARE ALREADY BEHIND the guy who built Claude Code at Anthropic said he stopped writing prompts a long time ago he runs loops instead, systems that write the prompts for him and keep working while he sleeps you type one request at a time and wait for the answer he built a machine that fires hundreds of requests, checks its own output and fixes itself with no hands on the keyboard here is how you make the jump step 1 stop treating the model like a chat box and start treating it like a worker you manage step 2 write a loop that takes a goal, breaks it into tasks and sends each one to the model on its own step 3 add a second pass where the model grades its own work and retries whatever failed step 4 let it run on a timer so it keeps shipping while you are away from the screen the person still typing prompts one by one is doing the work of a intern the person running loops is doing the work of a whole team off one laptop prompting was never the skill, building the system that prompts for you is
0:56

IF YOU ARE STILL PROMPTING BY HAND IN 2026 YOU ARE ALREADY BEHIND the guy who built Claude Code at Anthropic said he stopped writing prompts a long time ago he runs loops instead, systems that write the prompts for him and keep working while he sleeps you type one request at a time and wait for the answer he built a machine that fires hundreds of requests, checks its own output and fixes itself with no hands on the keyboard here is how you make the jump step 1 stop treating the model like a chat box and start treating it like a worker you manage step 2 write a loop that takes a goal, breaks it into tasks and sends each one to the model on its own step 3 add a second pass where the model grades its own work and retries whatever failed step 4 let it run on a timer so it keeps shipping while you are away from the screen the person still typing prompts one by one is doing the work of a intern the person running loops is doing the work of a whole team off one laptop prompting was never the skill, building the system that prompts for you is

Atlas

16,641 просмотров • 3 месяцев назад

Orchestrators vs. Graphs, clearly explained! orchestrators are great, and everyone builds one first. here is the ceiling: an orchestrator sits above the work and routes every message. five agents report to it. it reads all five. it decides what each one does next, and reads all five replies. that is ten trips through one context, and by the fifth agent that context has read four reports, five instructions and its own reasoning about all of them. Graph engineering fixes this by removing the seat: not a better router, but no router at all. you need both, and here is the sentence that resolves the whole confusion: an orchestrator sits above the work and holds all of it. a graph is the shape of the work, and holds none of it. ↳ above the work: one context that has to see everything before anything ships ↳ inside the work: a splitter that hands out and lets go, and a merge that reads nothing Prompts → Context → Harness → Loops → Graphs the coordination did not disappear. it moved into the edges, where it costs nothing and cannot get tired. the trick is noticing what you actually built. if one node has to see every result before the run can finish, you did not remove the bottleneck. you hired it, gave it the longest context in the system, and made it the thing you were counting on to stay sharp. one thing to know before you scale it. an orchestrator degrades in the one way nothing catches. ↳ it does not crash, time out or return an error. it stays up and keeps routing ↳ it just starts routing worse, somewhere around the fifth report, and every downstream agent does exactly what it was told that last one catches careful people. you can have perfect isolation on every worker and still have one window quietly drifting at the top, and the traces will all look clean because each worker did its job. and the one that eats whole nights: the merge is where this shows up first. ranking five findings is not judgment, it is a sort. if a model is doing it, you are paying a model to read five reports so it can put them in an order that three lines of code would have got right, and now that model has read everything too. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓
0:47

Orchestrators vs. Graphs, clearly explained! orchestrators are great, and everyone builds one first. here is the ceiling: an orchestrator sits above the work and routes every message. five agents report to it. it reads all five. it decides what each one does next, and reads all five replies. that is ten trips through one context, and by the fifth agent that context has read four reports, five instructions and its own reasoning about all of them. Graph engineering fixes this by removing the seat: not a better router, but no router at all. you need both, and here is the sentence that resolves the whole confusion: an orchestrator sits above the work and holds all of it. a graph is the shape of the work, and holds none of it. ↳ above the work: one context that has to see everything before anything ships ↳ inside the work: a splitter that hands out and lets go, and a merge that reads nothing Prompts → Context → Harness → Loops → Graphs the coordination did not disappear. it moved into the edges, where it costs nothing and cannot get tired. the trick is noticing what you actually built. if one node has to see every result before the run can finish, you did not remove the bottleneck. you hired it, gave it the longest context in the system, and made it the thing you were counting on to stay sharp. one thing to know before you scale it. an orchestrator degrades in the one way nothing catches. ↳ it does not crash, time out or return an error. it stays up and keeps routing ↳ it just starts routing worse, somewhere around the fifth report, and every downstream agent does exactly what it was told that last one catches careful people. you can have perfect isolation on every worker and still have one window quietly drifting at the top, and the traces will all look clean because each worker did its job. and the one that eats whole nights: the merge is where this shows up first. ranking five findings is not judgment, it is a sort. if a model is doing it, you are paying a model to read five reports so it can put them in an order that three lines of code would have got right, and now that model has read everything too. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓

Hanako

44,226 просмотров • 1 месяц назад

Another product weight scandal at Walmart Now you need to check your canned foods too Here are 2 cans of crushed pineapple, both are marked the same weight. Both have the water drained out of them One of the cans has a significant amount less of crushed pineapple compared to the other. The one that has more is from 8 months ago This isn’t Shrinkflation, this is fraud. Product weight issues happen far too often at Walmart to be an accident When this happens, report it to Weight and Measures commission
0:47

Another product weight scandal at Walmart Now you need to check your canned foods too Here are 2 cans of crushed pineapple, both are marked the same weight. Both have the water drained out of them One of the cans has a significant amount less of crushed pineapple compared to the other. The one that has more is from 8 months ago This isn’t Shrinkflation, this is fraud. Product weight issues happen far too often at Walmart to be an accident When this happens, report it to Weight and Measures commission

Wall Street Apes

46,201 просмотров • 6 месяцев назад

Jensen Huang told a room of global investors that AI is not one industry. It is five stacked on top of each other. Most people are investing in layer four and ignoring layers one through three entirely. He called it the five-layer cake. Layer one is energy. Jensen said this is the single greatest opportunity for the energy industry in a hundred years. The first time in a century that the grid in most countries can actually attract serious capital. Nuclear, solar, wind, hydrogen, it does not matter what form. If it produces energy, it gets funded. Siemens, GE Vernova, Mitsubishi. That is why they are all doing so well right now. Layer two is chips, computers, networking, and silicon photonics. Everything that processes the intelligence. Layer three is infrastructure. Land, power, buildings, data center operations. Every single one in short supply today. Layer four is the model layer. OpenAI, Anthropic. The layer everyone talks about. Layer five is applications. Every startup applying AI to financial services, legal, healthcare, logistics, transportation. Last year alone, a hundred billion dollars of venture capital went into this layer. The single largest VC year in the history of humanity. Then he said the number that stopped me cold. We are putting one trillion dollars into this five-layer cake this year. That sounds enormous. Jensen thinks the AI industry will eventually run at twenty trillion dollars per year. We are one trillion in of a twenty trillion dollar per year ecosystem. Most people watching AI are staring at layer four. Jensen was describing layers one through five as a single compounding system where every layer feeds the one above it. The people who understand that will invest differently than the people who do not.
4:50

Jensen Huang told a room of global investors that AI is not one industry. It is five stacked on top of each other. Most people are investing in layer four and ignoring layers one through three entirely. He called it the five-layer cake. Layer one is energy. Jensen said this is the single greatest opportunity for the energy industry in a hundred years. The first time in a century that the grid in most countries can actually attract serious capital. Nuclear, solar, wind, hydrogen, it does not matter what form. If it produces energy, it gets funded. Siemens, GE Vernova, Mitsubishi. That is why they are all doing so well right now. Layer two is chips, computers, networking, and silicon photonics. Everything that processes the intelligence. Layer three is infrastructure. Land, power, buildings, data center operations. Every single one in short supply today. Layer four is the model layer. OpenAI, Anthropic. The layer everyone talks about. Layer five is applications. Every startup applying AI to financial services, legal, healthcare, logistics, transportation. Last year alone, a hundred billion dollars of venture capital went into this layer. The single largest VC year in the history of humanity. Then he said the number that stopped me cold. We are putting one trillion dollars into this five-layer cake this year. That sounds enormous. Jensen thinks the AI industry will eventually run at twenty trillion dollars per year. We are one trillion in of a twenty trillion dollar per year ecosystem. Most people watching AI are staring at layer four. Jensen was describing layers one through five as a single compounding system where every layer feeds the one above it. The people who understand that will invest differently than the people who do not.

Ihtesham Ali

99,614 просмотров • 5 месяцев назад

here's how to build a game like this with Claude Opus 5.5 + Blender + Unity 6... the trick is preparing everything before Opus writes a single line, but there's a smart way to set it up: - step 1: connect Claude Code to Unity and Blender through MCP - step 2: write one big brief: the world, the mechanics, the UI, the mood, plus reference images - step 3: add a CLAUDE.md with your rules, and design docs as the single source of truth so Opus never guesses - step 4: split the work into subagents: design, modeling, textures, and one reviewer that only looks and approves - step 5: Opus models the assets in Blender: the island, palms, pier, boat, fishing hut - step 6: it builds a greybox in Unity first, plays it, fixes it, and repeats - step 7: mechanics one layer at a time: walking, casting, bait, catching, day and night - step 8: you watch and give feedback with a reason: "too dark at dusk", "cut the water stripes" and the two rules that make it work... the agent that builds is never the one that approves every fix gets written into a file, so the next project starts smarter start small, a fishing game is the perfect first project
0:52

here's how to build a game like this with Claude Opus 5.5 + Blender + Unity 6... the trick is preparing everything before Opus writes a single line, but there's a smart way to set it up: - step 1: connect Claude Code to Unity and Blender through MCP - step 2: write one big brief: the world, the mechanics, the UI, the mood, plus reference images - step 3: add a CLAUDE.md with your rules, and design docs as the single source of truth so Opus never guesses - step 4: split the work into subagents: design, modeling, textures, and one reviewer that only looks and approves - step 5: Opus models the assets in Blender: the island, palms, pier, boat, fishing hut - step 6: it builds a greybox in Unity first, plays it, fixes it, and repeats - step 7: mechanics one layer at a time: walking, casting, bait, catching, day and night - step 8: you watch and give feedback with a reason: "too dark at dusk", "cut the water stripes" and the two rules that make it work... the agent that builds is never the one that approves every fix gets written into a file, so the next project starts smarter start small, a fishing game is the perfect first project

rewind

187,520 просмотров • 1 месяц назад

Workflows vs. Graphs, clearly explained! workflows are great, and almost everyone has one. here is the ceiling: a workflow decides every step before it runs. you drew eight boxes in March. six months later the same three fire, every single time, and the other five have never once been reached. then a case arrives that nobody drew, and it goes to the closest wrong box. quietly, with a green status, because from the inside that looks exactly like success. Graph engineering fixes this by moving the decision: not what the steps do, but when the steps get chosen. you need both, and here is the sentence that resolves the whole confusion: a workflow decides the steps before it runs. a graph decides them while it runs. ↳ drawn in advance: the boxes, the branches, the order, the error path ↳ decided at runtime: how many units exist, what each one is allowed to see, which ones get created at all Prompts → Context → Harness → Loops → Graphs branches do not make it a graph. the branches were drawn in advance too, which means every one of them is a case you already thought of. the trick is knowing which part is allowed to be fixed. the node kinds are fixed. a splitter is a splitter, a gate is a gate, a merge is code. what is not fixed is how many of them exist this run, and that is decided after something has been read. one thing to know before you scale it. a workflow fails in a way that never pages anyone. ↳ the wrong branch ran, every check inside it passed, and the output is well formed ↳ nothing errored, because routing to the wrong box is not an error, it is a route that last one catches careful people. you cannot test your way out of it either, because the test suite was written from the same diagram that has the gap in it. and the one that eats whole nights: a workflow that has never surprised you is not stable, it is narrow. if it has run four hundred times and produced the same three shapes, it is not handling your work. it is handling the part of your work that fits it, and you have quietly stopped sending it the rest. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓
0:51

Workflows vs. Graphs, clearly explained! workflows are great, and almost everyone has one. here is the ceiling: a workflow decides every step before it runs. you drew eight boxes in March. six months later the same three fire, every single time, and the other five have never once been reached. then a case arrives that nobody drew, and it goes to the closest wrong box. quietly, with a green status, because from the inside that looks exactly like success. Graph engineering fixes this by moving the decision: not what the steps do, but when the steps get chosen. you need both, and here is the sentence that resolves the whole confusion: a workflow decides the steps before it runs. a graph decides them while it runs. ↳ drawn in advance: the boxes, the branches, the order, the error path ↳ decided at runtime: how many units exist, what each one is allowed to see, which ones get created at all Prompts → Context → Harness → Loops → Graphs branches do not make it a graph. the branches were drawn in advance too, which means every one of them is a case you already thought of. the trick is knowing which part is allowed to be fixed. the node kinds are fixed. a splitter is a splitter, a gate is a gate, a merge is code. what is not fixed is how many of them exist this run, and that is decided after something has been read. one thing to know before you scale it. a workflow fails in a way that never pages anyone. ↳ the wrong branch ran, every check inside it passed, and the output is well formed ↳ nothing errored, because routing to the wrong box is not an error, it is a route that last one catches careful people. you cannot test your way out of it either, because the test suite was written from the same diagram that has the gap in it. and the one that eats whole nights: a workflow that has never surprised you is not stable, it is narrow. if it has run four hundred times and produced the same three shapes, it is not handling your work. it is handling the part of your work that fits it, and you have quietly stopped sending it the rest. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓

Hanako

16,810 просмотров • 1 месяц назад

One prompt. One character. Everything. Every frame is code. Every asset is Layer. Claude Code wrote the video. Every asset in it came from Layer through the Layer MCP. Try it →
0:34

One prompt. One character. Everything. Every frame is code. Every asset is Layer. Claude Code wrote the video. Every asset in it came from Layer through the Layer MCP. Try it →

Layer

58,647 просмотров • 1 месяц назад

Elon Musk, absolute leader of the AI race with Grok Bot, and it's not a joke anymore. Ultimate guide on god-mode setup of Grok Bot, the org chart that runs while you sleep, step by step: A Chief of Staff sits in the middle with no tools of its own, BUT it reads the outcome you gave it, picks who does what, and never does the work itself. That one rule is why it never turns into the bottleneck you hired it to remove. → Researcher pulls real sources and tracks what's actually moving, not what sounds true → Writer turns that into finished copy while the research is still in the room → Visualiser gets three reference visuals once, then ships everything in that style forever → Analyst reads what performed and tells the rest of the team what to stop doing → Scheduler owns timing and holds the queue → Publisher actually ships What makes it different from every AI tool you've used: each bot gets its own computer in the cloud, its own browser, its own files, and they all share one memory. So the research is already sitting inside the draft before the draft starts. Nothing gets copy-pasted between tabs, nothing waits on you to approve step four of nine. And you never write a workflow for it. You hit record, do the job once the way you actually do it, stop. It pulls out the steps, saves them as a skill, and puts it on a schedule. The shape you're aiming for on every bot: everything reversible finished, nothing sent. 36 drafts queued, 0 published. It does all the work and stops dead at the one line only you can cross. You stop prompting. You start assigning. Full charter blocks, the approval line and the routines are in the article below ↓
0:24

Elon Musk, absolute leader of the AI race with Grok Bot, and it's not a joke anymore. Ultimate guide on god-mode setup of Grok Bot, the org chart that runs while you sleep, step by step: A Chief of Staff sits in the middle with no tools of its own, BUT it reads the outcome you gave it, picks who does what, and never does the work itself. That one rule is why it never turns into the bottleneck you hired it to remove. → Researcher pulls real sources and tracks what's actually moving, not what sounds true → Writer turns that into finished copy while the research is still in the room → Visualiser gets three reference visuals once, then ships everything in that style forever → Analyst reads what performed and tells the rest of the team what to stop doing → Scheduler owns timing and holds the queue → Publisher actually ships What makes it different from every AI tool you've used: each bot gets its own computer in the cloud, its own browser, its own files, and they all share one memory. So the research is already sitting inside the draft before the draft starts. Nothing gets copy-pasted between tabs, nothing waits on you to approve step four of nine. And you never write a workflow for it. You hit record, do the job once the way you actually do it, stop. It pulls out the steps, saves them as a skill, and puts it on a schedule. The shape you're aiming for on every bot: everything reversible finished, nothing sent. 36 drafts queued, 0 published. It does all the work and stops dead at the one line only you can cross. You stop prompting. You start assigning. Full charter blocks, the approval line and the routines are in the article below ↓

Miraqle

87,098 просмотров • 2 месяцев назад

ELON MUSK IS WRONG❗️ ISRAEL IS NOT #1 IN THE WORLD, HE IS. WE ARE #2. “I’m a huge admirer of the innovation coming out of Israel. It is objectively true that Israel punches high above its weight. I think, honestly, number one in the world. Innovation per capita, Israel is by far number one in the world.”
0:45

ELON MUSK IS WRONG❗️ ISRAEL IS NOT #1 IN THE WORLD, HE IS. WE ARE #2. “I’m a huge admirer of the innovation coming out of Israel. It is objectively true that Israel punches high above its weight. I think, honestly, number one in the world. Innovation per capita, Israel is by far number one in the world.”

Mossad Commentary

74,014 просмотров • 5 месяцев назад

China just released an open source AI model that matches the best closed models from OpenAI and Anthropic. Gavin Baker explained exactly how they did it and the answer should concern every American AI lab. The model is called GLM 5.2. It was built by Z. AI. You get 744 billion parameters, 1 million token context window and its MIT license, meaning anyone can download it, fork it, build a company on it, with no restrictions and no Dario. It scored 51 points on the artificial analysis intelligence index. The highest score any open weight model has ever achieved. It beat GPT 5.5 on the frontier software engineering benchmark. It trails Claude Opus 4.8 by less than one percentage point. And it costs 85% less to run than GPT 5.5 for comparable performance. Gavin Baker said on the All-In podcast that this model has challenged some of his beliefs. Then he explained how China built it. The method is called distillation. Just think of tens of thousands of phones and computers running simultaneously, all hitting the frontier model APIs through masked accounts, asking specific questions, and harvesting what happens inside the model when it answers. Every reasoning step, every token. The entire thinking process gets recorded and fed back into the Chinese model during training. It is a cheat sheet. It is the answer key to the exam. And here is the part that should worry everyone. Sacks said it plainly. China was already nine months behind American models. But now that GLM 5.2 is good enough to run its own reinforcement learning, it can improve itself without needing to distill from American models anymore. The cheat sheet let them get close enough to start writing their own answers. Sacks said we are six months behind on the model and 24 months behind on silicon and they are only a few months behind in total. The Z. AI founder told Elon Musk directly that open weight fable-level capability will be here before Q1 2027. Every restriction Anthropic lobbied for, every self-imposed safety guardrail, every month of delay in releasing American frontier models accelerated this. The Chinese labs were not under those restrictions. They were not going to wait. The composable model future Gavin described, where every enterprise runs a frontier model alongside their own fine-tuned open weight model, is coming regardless of what American labs do next. The question is just whether the open weight half of that stack is American or Chinese. Right now it is Chinese. WATCH THE FULL PODCAST ON The All-In Podcast
3:21

China just released an open source AI model that matches the best closed models from OpenAI and Anthropic. Gavin Baker explained exactly how they did it and the answer should concern every American AI lab. The model is called GLM 5.2. It was built by Z. AI. You get 744 billion parameters, 1 million token context window and its MIT license, meaning anyone can download it, fork it, build a company on it, with no restrictions and no Dario. It scored 51 points on the artificial analysis intelligence index. The highest score any open weight model has ever achieved. It beat GPT 5.5 on the frontier software engineering benchmark. It trails Claude Opus 4.8 by less than one percentage point. And it costs 85% less to run than GPT 5.5 for comparable performance. Gavin Baker said on the All-In podcast that this model has challenged some of his beliefs. Then he explained how China built it. The method is called distillation. Just think of tens of thousands of phones and computers running simultaneously, all hitting the frontier model APIs through masked accounts, asking specific questions, and harvesting what happens inside the model when it answers. Every reasoning step, every token. The entire thinking process gets recorded and fed back into the Chinese model during training. It is a cheat sheet. It is the answer key to the exam. And here is the part that should worry everyone. Sacks said it plainly. China was already nine months behind American models. But now that GLM 5.2 is good enough to run its own reinforcement learning, it can improve itself without needing to distill from American models anymore. The cheat sheet let them get close enough to start writing their own answers. Sacks said we are six months behind on the model and 24 months behind on silicon and they are only a few months behind in total. The Z. AI founder told Elon Musk directly that open weight fable-level capability will be here before Q1 2027. Every restriction Anthropic lobbied for, every self-imposed safety guardrail, every month of delay in releasing American frontier models accelerated this. The Chinese labs were not under those restrictions. They were not going to wait. The composable model future Gavin described, where every enterprise runs a frontier model alongside their own fine-tuned open weight model, is coming regardless of what American labs do next. The question is just whether the open weight half of that stack is American or Chinese. Right now it is Chinese. WATCH THE FULL PODCAST ON The All-In Podcast

Ihtesham Ali

86,957 просмотров • 4 месяцев назад

We often hear that Machine Learning models learn patterns in data. But what does that actually look like in Geometry? If you dropped a little elastic mesh into a cloud of points and let it learn, how would it fold itself to match the shape of the data? In this scene we watch a Self-Organizing Map (SOM), a simple unsupervised neural model, learn the shape of a 3D datasets l, one static and the other dynamic. On top of this, we lay down a square grid of neurons whose weights live in the same plane. At the start, this grid is just a flat net floating across the cloud. It knows nothing about the structure underneath. Learning is a repeated game: Pick a random data point, find the neuron whose weight is closest, and then nudge that neuron and its neighbours toward the point. Do this again and again, while slowly shrinking how far the neighbourhood influence spreads. Python code is available for Subscribers. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML
0:57

We often hear that Machine Learning models learn patterns in data. But what does that actually look like in Geometry? If you dropped a little elastic mesh into a cloud of points and let it learn, how would it fold itself to match the shape of the data? In this scene we watch a Self-Organizing Map (SOM), a simple unsupervised neural model, learn the shape of a 3D datasets l, one static and the other dynamic. On top of this, we lay down a square grid of neurons whose weights live in the same plane. At the start, this grid is just a flat net floating across the cloud. It knows nothing about the structure underneath. Learning is a repeated game: Pick a random data point, find the neuron whose weight is closest, and then nudge that neuron and its neighbours toward the point. Do this again and again, while slowly shrinking how far the neighbourhood influence spreads. Python code is available for Subscribers. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML

Mathelirium

76,377 просмотров • 8 месяцев назад