Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Your brain doesn't search through memories — it falls downhill on an energy landscape and lands in the nearest valley. This is the Hopfield network from 1982, and the same math now lives inside every transformer: the attention mechanism in modern LLMs is mathematically equivalent to a Hopfield network... with exponential storage capacity. The idea: neurons are +1 or -1, connected by symmetric weights. Memories are local minima sculpted by Hebbian learning — neurons that fire together, wire together. Give the network a noisy input, and it descends toward the nearest stored memory one GPU-style majority vote at a time. Vanilla Hopfield nets cap at ~14% of neuron count as usable memories, but the modern extension blows past that. 23 minutes. Bookmark & watch today.show more

Eron

6,987 subscribers

22,378 Aufrufe • vor 10 Tagen •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Filip M Madsen, PhD | Molecular Dominoes

Filip M Madsen, PhD | Molecular Dominoesvor 9 Tagen

Yes, a mathematician almost disproved life once because of this, using protein chemistry. Turns out all "life" = "falling towards the lowest state".

Profilbild von Lilith Datura

Lilith Daturavor 10 Tagen

True

Profilbild von บีมน้อย บีมน้อย

บีมน้อย บีมน้อยvor 10 Tagen

A mature approach to artificial intelligence balances technical performance with ethical considerations in large-scale production environments.

Profilbild von Elizabeth

Elizabethvor 9 Tagen

Energy landscape is correct. But it is not just the nearest. It is the deepest and largest cluster

Profilbild von Turner NextGen AI

Turner NextGen AIvor 10 Tagen

That’s not how intelligence should be represented. Because you have nothing to do with Movement sciences, and this is the biggest problem with using an LLM versus a movement based AI system an LLM system for whoever reason put the lexicons in under motion, sciences, not movement sciences so by mechanics is not an action or a functional movement. It is just a representation of a body in space so the fact that that LLM is not anchored in any kind of developmental sciences movement science biology for function you’re asking for intelligence to represent and reason off of something that can’t be reason and that’s where you get the hallucinations within the system and they can’t scale to the way that we want them to. I left an LLM background for my AI two years ago, but I’m still dealing with these lexicons or taxonomies.

Profilbild von Sun&Steelx

Sun&Steelxvor 10 Tagen

Pancomputationalism...

Profilbild von Sonechka 🌹🐻‍❄️7Сонечка

Sonechka 🌹🐻‍❄️7Сонечкаvor 10 Tagen

nothing lost everything is recorded from a somatic cell in the womb. our brain frequencies hop indeed.

Profilbild von Bernhard Mueller

Bernhard Muellervor 9 Tagen

That's exactly our implementation:

Profilbild von Colin Johnson

Colin Johnsonvor 9 Tagen

Correct.

Profilbild von nceladus

nceladusvor 10 Tagen

fascinating

Ähnliche Videos

I GAVE A DRONE THE BRAIN OF A FRUIT FLY Not a toy network with a handful of fake neurons. The actual wiring diagram of a real fly. 166,700 neurons, 25.6 million connections, mapped one synapse at a time. I call it FlyDrones. The drone's camera becomes the fly's eyes. Motion lights up the same neurons a real fly uses to see. Spikes race through the connectome down to the neurons that drive the wings. Those spikes become throttle and yaw. The brain has zero lines of flight code, just wiring. Swing a fly swatter at it. If the giant fiber (the fly's escape neuron) fires in time, the drone dodges. Open palm, fist, rush at the camera and it reacts like a fly. Built for Tello, Crazyflie, ArduPilot/PX4 and Betaflight quads via ESP32. Code is MIT, powered by Claude. Fork it and give your drone a brain.

I GAVE A DRONE THE BRAIN OF A FRUIT FLY Not a toy network with a handful of fake neurons. The actual wiring diagram of a real fly. 166,700 neurons, 25.6 million connections, mapped one synapse at a time. I call it FlyDrones. The drone's camera becomes the fly's eyes. Motion lights up the same neurons a real fly uses to see. Spikes race through the connectome down to the neurons that drive the wings. Those spikes become throttle and yaw. The brain has zero lines of flight code, just wiring. Swing a fly swatter at it. If the giant fiber (the fly's escape neuron) fires in time, the drone dodges. Open palm, fist, rush at the camera and it reacts like a fly. Built for Tello, Crazyflie, ArduPilot/PX4 and Betaflight quads via ESP32. Code is MIT, powered by Claude. Fork it and give your drone a brain.

Spike 1%

15,703 Aufrufe • vor 16 Tagen

Every quantum state in the universe is a vector - not in 3D space, but in a complex space with potentially infinite dimensions where the sum of squared coefficients always equals exactly 1. That constraint is the same math LLMs use to normalize probability distributions over tokens. A wavefunction collapsing to a measurement and a softmax choosing the next word are the same operation - projecting an infinite-dimensional vector onto a basis. Claude and GPT do quantum math on every forward pass. 14 minutes. Bookmark & watch today. Quantum mechanics and LLMs speak the same language.

Every quantum state in the universe is a vector - not in 3D space, but in a complex space with potentially infinite dimensions where the sum of squared coefficients always equals exactly 1. That constraint is the same math LLMs use to normalize probability distributions over tokens. A wavefunction collapsing to a measurement and a softmax choosing the next word are the same operation - projecting an infinite-dimensional vector onto a basis. Claude and GPT do quantum math on every forward pass. 14 minutes. Bookmark & watch today. Quantum mechanics and LLMs speak the same language.

Eron

18,166 Aufrufe • vor 1 Monat

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

39,391 Aufrufe • vor 11 Monaten

the power of choice (explained by neuroscience): your brain is a creature of repetition. it thrives on patterns, predictability, rhythm. every time you repeat a thought, action, or behavior, your neurons fire the same sequence. do that enough and you've built a pathway. neuroscience calls this hebbian learning: neurons that fire together, wire together. it's your brain's way of cutting corners. that's why brushing your teeth or checking your phone or going for a walk stops taking effort. it just... happens. here's the wild part, your brain doesn't care if the habit is good for you or not. it doesn't sort "healthy" from "harmful." it just tracks what you repeat. so stress-scrolling and morning journaling get treated the exact same way by your nervous system: familiar. and familiar reads as safe. which means every small, boring, repeated action is quietly rewiring you. for ease. for self-trust. so next time a new habit feels awkward or uncomfortable, that's not failure. that's your brain being asked to trust a new pattern instead of the old one. it takes time. it takes regulation. it takes patience with yourself. that's also why nervous system regulation matters so much for habit change. when your body feels safe, your prefrontal cortex, the part that handles focus and decisions, comes back online. that's the moment real change actually starts.

the power of choice (explained by neuroscience): your brain is a creature of repetition. it thrives on patterns, predictability, rhythm. every time you repeat a thought, action, or behavior, your neurons fire the same sequence. do that enough and you've built a pathway. neuroscience calls this hebbian learning: neurons that fire together, wire together. it's your brain's way of cutting corners. that's why brushing your teeth or checking your phone or going for a walk stops taking effort. it just... happens. here's the wild part, your brain doesn't care if the habit is good for you or not. it doesn't sort "healthy" from "harmful." it just tracks what you repeat. so stress-scrolling and morning journaling get treated the exact same way by your nervous system: familiar. and familiar reads as safe. which means every small, boring, repeated action is quietly rewiring you. for ease. for self-trust. so next time a new habit feels awkward or uncomfortable, that's not failure. that's your brain being asked to trust a new pattern instead of the old one. it takes time. it takes regulation. it takes patience with yourself. that's also why nervous system regulation matters so much for habit change. when your body feels safe, your prefrontal cortex, the part that handles focus and decisions, comes back online. that's the moment real change actually starts.

alexei

16,524 Aufrufe • vor 19 Tagen

Wow! This Changes Everything We Thought We Knew About Memory It is a groundbreaking big deal. TOU ARE MAKING GENERATIONAL MEMORIES RIGHT NOW IN EACH CELL! Scientists just found that your brain doesn’t just store memories — it stores the rules for how those memories will change in the future. A brand-new preprint from Stanford’s Greenleaf and Schnitzer labs (led by PhD student Yuxi Ke) drops a bombshell that feels like science fiction becoming reality overnight. For decades, neuroscientists have suspected that chromatin the DNA packaging material inside every cell nucleus might somehow “store” memory-related information. But what kind of information? Content? Timing? Rules? Now we have the answer! Using activity-dependent genetic tagging, fear conditioning, and single-nucleus multiome sequencing in the mouse medial prefrontal cortex (the brain’s long-term memory vault), the team tracked engram neurons for a full month after a memory was formed. What they discovered is electric: - One month after encoding, engram neurons have acquired a completely new chromatin landscape. - These chromatin changes are almost invisible at 7 days… but roar into existence by 28 days. - At recall, these engram cells don’t just “remember” better — they rewrite their entire transcriptional response. They preferentially fire up chromatin regulators, RNA processing machinery, and protein-turnover systems instead of simply boosting classic plasticity genes. In other words: the chromatin doesn’t just hold the memory of the past. It holds metaplastic instructions— rules that dictate how the neuron will respond the next time the memory is triggered. They call it chromatin metaplasticity. This is the “future tense of memory.” It is a massive deal 1. Memory is not just synapses. For 70+ years we’ve been obsessed with synaptic weights. This work proves the nucleus itself is a computational device that stores history-dependent rules. 2. It explains remote memory. The chromatin signature keeps maturing for weeks after the experience, perfectly matching the time course of systems consolidation into the cortex. 3. It’s energy-efficient genius. Instead of constantly maintaining memory proteins, the cell stores a silent, writable program that only activates when needed. Nature’s version of lazy evaluation. 4. It links development to adult memory. The late chromatin state is enriched for the exact same transcription-factor motifs used in embryonic development. Your adult brain is still running developmental software to lock in lifelong memories. 5. Huge therapeutic potential. If we can read or rewrite these chromatin metaplastic rules, we might one day boost failing remote memories in Alzheimer’s… or selectively dampen traumatic ones. This isn’t incremental. But a brand new layer of the memory code. And it explains WHY a person can receive memory from an organ transplant. It also explains generational traumas. Link: The future of neuroscience just got a lot more exciting and a lot more nuclear. Your chromatin is writing tomorrow’s memories today. And we finally have the first page of the instruction manual.

Brian Roemmele

45,345 Aufrufe • vor 1 Monat

166,700 fruit fly neurons are walking a robot cat across a desk right now. In September 2026, Google Research and HHMI Janelia mapped every neuron and every connection in an adult male fruit fly's full nervous system, brain, optic lobes, and ventral nerve cord together, and published it in Cell. The number is staggering on its own: 166,700 neurons, roughly 125 million synapses, the most complete nervous system map of any animal ever built. Now that exact wiring is running live, simulated neuron for neuron, inside a small robotic quadruped equipped with a camera and motion sensors. Movement comes in through the sensors, the biological circuitry fires the way it always did, and the output drives the robot's legs directly. No one wrote the balance logic. No one coded the gait. The insect's own nervous system does that job, just on legs it never had. To be clear about what this is and isn't: nothing biological is alive in this loop. It's a full computational reconstruction of real neural wiring, controlling hardware instead of an insect body. The brain is real. The body is borrowed. Remarkable proof that a mapped nervous system can drive any body you give it, or a preview of something people aren't ready to think through yet?

166,700 fruit fly neurons are walking a robot cat across a desk right now. In September 2026, Google Research and HHMI Janelia mapped every neuron and every connection in an adult male fruit fly's full nervous system, brain, optic lobes, and ventral nerve cord together, and published it in Cell. The number is staggering on its own: 166,700 neurons, roughly 125 million synapses, the most complete nervous system map of any animal ever built. Now that exact wiring is running live, simulated neuron for neuron, inside a small robotic quadruped equipped with a camera and motion sensors. Movement comes in through the sensors, the biological circuitry fires the way it always did, and the output drives the robot's legs directly. No one wrote the balance logic. No one coded the gait. The insect's own nervous system does that job, just on legs it never had. To be clear about what this is and isn't: nothing biological is alive in this loop. It's a full computational reconstruction of real neural wiring, controlling hardware instead of an insect body. The brain is real. The body is borrowed. Remarkable proof that a mapped nervous system can drive any body you give it, or a preview of something people aren't ready to think through yet?

Zentrix⌚️

231,611 Aufrufe • vor 17 Tagen

🤯 Your entire memory might NOT be inside your brain. We live in a Holographic Universe 😱 Neuropsychiatrist Diane Hennacy Powell of the just dropped this on Danny Jones Podcast “Memory is stored in a field — not in the brain. We’ve already proven it with rats.” Lashley cut huge parts of rats’ brains… and they still remembered mazes perfectly. Brain damage doesn’t erase memories the way we expect. What if your brain is just the TV — and the show is being broadcast from somewhere else? Holographic universe theory says every piece contains the whole. So… where are YOUR memories actually stored? ↓ Watch the clip — what do you think after seeing this? #Consciousness #Neuroscience #HolographicUniverse

🤯 Your entire memory might NOT be inside your brain. We live in a Holographic Universe 😱 Neuropsychiatrist Diane Hennacy Powell of the just dropped this on Danny Jones Podcast “Memory is stored in a field — not in the brain. We’ve already proven it with rats.” Lashley cut huge parts of rats’ brains… and they still remembered mazes perfectly. Brain damage doesn’t erase memories the way we expect. What if your brain is just the TV — and the show is being broadcast from somewhere else? Holographic universe theory says every piece contains the whole. So… where are YOUR memories actually stored? ↓ Watch the clip — what do you think after seeing this? #Consciousness #Neuroscience #HolographicUniverse

Interstellar

35,520 Aufrufe • vor 8 Monaten

Dropout by hand ✍️ ~ 10 steps walkthrough below Dropout is the simplest trick in deep learning that actually works: during training you randomly switch neurons off, so the network cannot lean on any one of them. It is two lines of code and almost nobody has worked through what those lines do to the numbers. So I drew and calculated one entirely by hand. Goal: train one pass through a small network with two dropout layers, then run inference with dropout switched off. The network: Linear(2,4), ReLU, Dropout(0.5), Linear(4,3), ReLU, Dropout(0.33), Linear(3,2). = 1. Given = A training set of two examples, X1 and X2, and the weight matrices for all three linear layers. = 2. Draw the first random numbers = Let us draw 4 random numbers, one per neuron in the first hidden layer. Above 0.5 we keep (◯), below we drop (╳). Here that gives [◯, ╳, ◯, ╳]. = 3. Build the first dropout matrix = We turn that pattern into a diagonal matrix. The scaling factor is 1/(1-p) = 2, so a kept neuron gets 2 and a dropped one gets 0. Multiplying by it does both jobs at once: it deletes the 2nd and 4th neurons and doubles the two that survive. = 4. Draw the second random numbers = Let us do it again for the 3 neurons in the next layer, this time against p = 0.33. The result is [◯, ◯, ╳]. = 5. Build the second dropout matrix = We set the diagonal to 1.5 where kept and 0 where dropped. Only the 3rd neuron goes. = 6. Feed forward = Let us run the whole thing top to bottom: one matrix multiplication per layer, ReLU setting the negatives to zero, and the two dropout matrices doing their work in between. The outputs Y come out at the bottom. = 7. MSE loss gradients = We compare Y against the targets Y', subtract, and multiply each element by 2. That is the whole gradient of the mean squared error. = 8. Update the weights = Let us push those gradients back through the network and update the weights (marked in light red). = 9. Deactivate dropout = Training is over, so we set both dropout matrices to the identity. Every neuron is back, and nothing is scaled. = 10. Feed forward again = One more pass, this time on unseen data, to make the prediction. You have just trained and run a network with dropout by hand. ✍️ The outputs: Training outputs Y = [-6, 9; 13, 4] Loss gradients = [-4, 4; 6, -2] Inference outputs = [13, 13; 4, 3] 💾 Save this post! #AIbyHand #Dropout #DeepLearning #NeuralNetworks

Dropout by hand ✍️ ~ 10 steps walkthrough below Dropout is the simplest trick in deep learning that actually works: during training you randomly switch neurons off, so the network cannot lean on any one of them. It is two lines of code and almost nobody has worked through what those lines do to the numbers. So I drew and calculated one entirely by hand. Goal: train one pass through a small network with two dropout layers, then run inference with dropout switched off. The network: Linear(2,4), ReLU, Dropout(0.5), Linear(4,3), ReLU, Dropout(0.33), Linear(3,2). = 1. Given = A training set of two examples, X1 and X2, and the weight matrices for all three linear layers. = 2. Draw the first random numbers = Let us draw 4 random numbers, one per neuron in the first hidden layer. Above 0.5 we keep (◯), below we drop (╳). Here that gives [◯, ╳, ◯, ╳]. = 3. Build the first dropout matrix = We turn that pattern into a diagonal matrix. The scaling factor is 1/(1-p) = 2, so a kept neuron gets 2 and a dropped one gets 0. Multiplying by it does both jobs at once: it deletes the 2nd and 4th neurons and doubles the two that survive. = 4. Draw the second random numbers = Let us do it again for the 3 neurons in the next layer, this time against p = 0.33. The result is [◯, ◯, ╳]. = 5. Build the second dropout matrix = We set the diagonal to 1.5 where kept and 0 where dropped. Only the 3rd neuron goes. = 6. Feed forward = Let us run the whole thing top to bottom: one matrix multiplication per layer, ReLU setting the negatives to zero, and the two dropout matrices doing their work in between. The outputs Y come out at the bottom. = 7. MSE loss gradients = We compare Y against the targets Y', subtract, and multiply each element by 2. That is the whole gradient of the mean squared error. = 8. Update the weights = Let us push those gradients back through the network and update the weights (marked in light red). = 9. Deactivate dropout = Training is over, so we set both dropout matrices to the identity. Every neuron is back, and nothing is scaled. = 10. Feed forward again = One more pass, this time on unseen data, to make the prediction. You have just trained and run a network with dropout by hand. ✍️ The outputs: Training outputs Y = [-6, 9; 13, 4] Loss gradients = [-4, 4; 6, -2] Inference outputs = [13, 13; 4, 3] 💾 Save this post! #AIbyHand #Dropout #DeepLearning #NeuralNetworks

Tom Yeh

14,442 Aufrufe • vor 2 Monaten

Most people first see Euler’s Formula as a strange equation in a textbook. Then years later, they realize it quietly powers the modern world. Leonhard Euler discovered that these seemingly unrelated mathematical ideas: → exponential growth → imaginary numbers → sine waves → cosine waves → rotation … are all deeply connected through one identity: e^(iθ) = cos(θ) + i·sin(θ) At first glance it looks impossible. How can an exponential function suddenly produce circles and waves? The key insight is that multiplying by a complex exponential creates rotation. As the angle θ changes: - the cosine term tracks horizontal motion - the sine term tracks vertical motion - together they trace a perfect circle in the complex plane Euler showed that waves and rotation are mathematically the same phenomenon viewed differently. That single idea changed science and engineering forever. Today, Euler’s Formula sits underneath: → Fourier Transforms → signal processing → wireless communication → MRI scanners → quantum mechanics → electrical engineering → audio compression → radar systems → GPS → neural network frequency analysis Even modern AI systems indirectly rely on mathematics built on top of these foundations. The famous special case is Euler’s Identity: e^(iπ) + 1 = 0 Richard Feynman reportedly called it: "our jewel." Because it revealed that mathematics is not a collection of separate topics. It is one connected language describing reality itself.

Most people first see Euler’s Formula as a strange equation in a textbook. Then years later, they realize it quietly powers the modern world. Leonhard Euler discovered that these seemingly unrelated mathematical ideas: → exponential growth → imaginary numbers → sine waves → cosine waves → rotation … are all deeply connected through one identity: e^(iθ) = cos(θ) + i·sin(θ) At first glance it looks impossible. How can an exponential function suddenly produce circles and waves? The key insight is that multiplying by a complex exponential creates rotation. As the angle θ changes: - the cosine term tracks horizontal motion - the sine term tracks vertical motion - together they trace a perfect circle in the complex plane Euler showed that waves and rotation are mathematically the same phenomenon viewed differently. That single idea changed science and engineering forever. Today, Euler’s Formula sits underneath: → Fourier Transforms → signal processing → wireless communication → MRI scanners → quantum mechanics → electrical engineering → audio compression → radar systems → GPS → neural network frequency analysis Even modern AI systems indirectly rely on mathematics built on top of these foundations. The famous special case is Euler’s Identity: e^(iπ) + 1 = 0 Richard Feynman reportedly called it: "our jewel." Because it revealed that mathematics is not a collection of separate topics. It is one connected language describing reality itself.

Tech with Mak

25,504 Aufrufe • vor 4 Monaten

Renaissance founding president and B Capital chairman Howard L Morgan says SpaceX has a monopoly, but it doesn't benefit from network effects like Google or Facebook. "Right now, SpaceX's launch monopoly is a real monopoly, pretty much." "But little bits are being chipped away at the edge. And I think when Blue Origin gets it together, they'll be big. Because everybody wants to go elsewhere." "It's different than Google's search, where you had real network effects. There aren't real network effects in launch. And so they don't get the benefit of that." "Facebook got the benefit of network effects. Google got the benefit of network effects. SpaceX doesn't have the benefit of network effects." "And even in the data center world, there's no real network effect benefit. So there is room for competition."

Renaissance founding president and B Capital chairman Howard L Morgan says SpaceX has a monopoly, but it doesn't benefit from network effects like Google or Facebook. "Right now, SpaceX's launch monopoly is a real monopoly, pretty much." "But little bits are being chipped away at the edge. And I think when Blue Origin gets it together, they'll be big. Because everybody wants to go elsewhere." "It's different than Google's search, where you had real network effects. There aren't real network effects in launch. And so they don't get the benefit of that." "Facebook got the benefit of network effects. Google got the benefit of network effects. SpaceX doesn't have the benefit of network effects." "And even in the data center world, there's no real network effect benefit. So there is room for competition."

Jack Kuhr

10,324 Aufrufe • vor 3 Monaten

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Alex Ker 🔭

1,367,117 Aufrufe • vor 3 Jahren

Backpropagation by hand ✍️ ~ 11 steps walkthrough below Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time. So I drew and calculated one entirely by hand. Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias. = 1. Given = A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0]. = 2. Backprop gradient cells = Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first. = 3. Layer 3 softmax = We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired. = 4. Layer 3 weights and biases = Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together. = 5. Layer 2 activations = We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward. = 6. Layer 2 ReLU = Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else. = 7. Layer 2 weights and biases = We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up. = 8. Layer 1 activations = Let us multiply dL/dz2 by W2. = 9. Layer 1 ReLU = We apply the same gate again, now on a1. = 10. Layer 1 weights and biases = Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient. = 11. Update = We subtract, and the network has learned. In practice a learning rate scales this step. The gradients: dL/dz3 = [0.5, -0.5, 0] dL/da1 = [1, -2, 2, -1] dL/dz1 = [0, -2, 2, -1] The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning. 💾 Save this post!

Backpropagation by hand ✍️ ~ 11 steps walkthrough below Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you cannot do. It is matrix multiplication, working backward, one layer at a time. So I drew and calculated one entirely by hand. Goal: push the loss gradient back through a 3-layer network and land on a new value for every weight and bias. = 1. Given = A 3-layer perceptron, an input X, predictions Ypred = [0.5, 0.5, 0], and the truth Ytarget = [0, 1, 0]. = 2. Backprop gradient cells = Let us draw empty cells for every gradient we are about to compute. The shape of the answer comes first. = 3. Layer 3 softmax = We get dL/dz3 straight from Ypred minus Ytarget = [0.5, -0.5, 0]. No chain rule needed, and that shortcut is the whole reason softmax and cross-entropy are paired. = 4. Layer 3 weights and biases = Let us multiply dL/dz3 by [a2 | 1]. One multiplication gives the gradient for W3 and b3 together. = 5. Layer 2 activations = We multiply dL/dz3 by W3 to get dL/da2. The gradient moves back across a layer the same way the signal moved forward. = 6. Layer 2 ReLU = Let us pass it through the gate: keep the gradient where the activation was positive, zero it everywhere else. = 7. Layer 2 weights and biases = We multiply dL/dz2 by [a1 | 1]. The same figure as step 4, one layer up. = 8. Layer 1 activations = Let us multiply dL/dz2 by W2. = 9. Layer 1 ReLU = We apply the same gate again, now on a1. = 10. Layer 1 weights and biases = Let us multiply dL/dz1 by [x | 1], and every weight in the network now has a gradient. = 11. Update = We subtract, and the network has learned. In practice a learning rate scales this step. The gradients: dL/dz3 = [0.5, -0.5, 0] dL/da1 = [1, -2, 2, -1] dL/dz1 = [0, -2, 2, -1] The takeaway: matrix multiplication is all you need. Just like the forward pass, backpropagation is matrix multiplications end to end. You can do every one by hand, slowly and imperfectly, which is exactly why a GPU's ability to do them fast mattered so much to deep learning. 💾 Save this post!

Tom Yeh

962,430 Aufrufe • vor 2 Monaten

A PHYSICS PROFESSOR GAVE THE SAME LECTURE 1,742 TIMES TO WARN ABOUT ONE IDEA QUIETLY BREAKING THE HUMAN RACE. IT IS THE WORD EVERY AI LAB THROWS AROUND TODAY AND ALMOST NONE OF THEM ACTUALLY UNDERSTAND IT. 72 minutes from Albert Bartlett, a University of Colorado physicist, on the one piece of math he called humanity's greatest blind spot. -> The idea that lands: a small steady percentage is never small. 7% a year sounds calm, but it doubles in a decade, doubles again in the next, and quietly runs away from you. He shows it with bacteria in a bottle, with money, with whole cities -- the same curve that looks flat for ages and then explodes almost overnight. Everyone calls that curve "Exponential". He proves most people hear it as "Fast" when it actually means something far more violent. Right now every lab and every headline calls AI exponential. This is the talk that shows you what the word truly does -- so you see both the hype and the real danger the way the math demands. You thought it was a boring math word. It was the whole story the entire time. Save this one. You'll never hear "Exponential" the same way ↓

A PHYSICS PROFESSOR GAVE THE SAME LECTURE 1,742 TIMES TO WARN ABOUT ONE IDEA QUIETLY BREAKING THE HUMAN RACE. IT IS THE WORD EVERY AI LAB THROWS AROUND TODAY AND ALMOST NONE OF THEM ACTUALLY UNDERSTAND IT. 72 minutes from Albert Bartlett, a University of Colorado physicist, on the one piece of math he called humanity's greatest blind spot. -> The idea that lands: a small steady percentage is never small. 7% a year sounds calm, but it doubles in a decade, doubles again in the next, and quietly runs away from you. He shows it with bacteria in a bottle, with money, with whole cities -- the same curve that looks flat for ages and then explodes almost overnight. Everyone calls that curve "Exponential". He proves most people hear it as "Fast" when it actually means something far more violent. Right now every lab and every headline calls AI exponential. This is the talk that shows you what the word truly does -- so you see both the hype and the real danger the way the math demands. You thought it was a boring math word. It was the whole story the entire time. Save this one. You'll never hear "Exponential" the same way ↓

slash1s

100,138 Aufrufe • vor 3 Monaten

What if you could visualise how electricity flows from a wind farm in Scotland to your home in London? Ok it's not quite that simple, but it got me thinking that there must be a way of visualising the complexity of the electricity transmission grid in Great Britain. This is my first attempt… What you're seeing are the real power lines and substations on the transmission grid using OpenStreetMap data. The real connections between each substation are found using PyPSA-Eur as well as calculating the "electrical distance" or impedance between each substation. This is my incredibly basic and naive way of attempting to show the "speed" at which the energy is moving through the network, where the bottlenecks are, etc. I know it's not as simple as this, but it's a first attempt so… 😉 I've also added a way to pick any substation in the network and show how electricity pulses through the rest of the network from that point. This is the magic sauce that really brings to life how much infrastructure electricity has to pass through to get from point A to point B. Of course this isn't really what happens, and there's no demand to soak up the energy. However I think it still serves as an interesting visual of how complex the network actually is and the distances involved to get energy from one side of the country to the other. What do you think? Is this worth exploring further?

What if you could visualise how electricity flows from a wind farm in Scotland to your home in London? Ok it's not quite that simple, but it got me thinking that there must be a way of visualising the complexity of the electricity transmission grid in Great Britain. This is my first attempt… What you're seeing are the real power lines and substations on the transmission grid using OpenStreetMap data. The real connections between each substation are found using PyPSA-Eur as well as calculating the "electrical distance" or impedance between each substation. This is my incredibly basic and naive way of attempting to show the "speed" at which the energy is moving through the network, where the bottlenecks are, etc. I know it's not as simple as this, but it's a first attempt so… 😉 I've also added a way to pick any substation in the network and show how electricity pulses through the rest of the network from that point. This is the magic sauce that really brings to life how much infrastructure electricity has to pass through to get from point A to point B. Of course this isn't really what happens, and there's no demand to soak up the energy. However I think it still serves as an interesting visual of how complex the network actually is and the distances involved to get energy from one side of the country to the other. What do you think? Is this worth exploring further?

Robin Hawkes

144,544 Aufrufe • vor 8 Monaten

Watch the Black Bull grow a brain! 🐂🀄️🧠 The first memecoin Clone co-created by its community and made tangible. Clude CludeAI turns every solana:9cRCn9rGT8V2imeM2BaKs13yhMEais3ruM3rPvTGpump post into a tokenized memory on Solana, feeding the live network behind the Ansem Clone. 165K memories now shine as stars and growing in real time. Brightest = most liked. Find yours: Ansem 🐂🀄️ is there a Black Bull Builders grant we can apply for to scale this into the first MMO AI with billions of tokenized memories feeding a live community hivemind?

Watch the Black Bull grow a brain! 🐂🀄️🧠 The first memecoin Clone co-created by its community and made tangible. Clude CludeAI turns every solana:9cRCn9rGT8V2imeM2BaKs13yhMEais3ruM3rPvTGpump post into a tokenized memory on Solana, feeding the live network behind the Ansem Clone. 165K memories now shine as stars and growing in real time. Brightest = most liked. Find yours: Ansem 🐂🀄️ is there a Black Bull Builders grant we can apply for to scale this into the first MMO AI with billions of tokenized memories feeding a live community hivemind?

seb

191,552 Aufrufe • vor 2 Monaten

NSA WHISTLEBLOWER REACTS TO LG TV SPYWARE INVESTIGATION, WARNS ABOUT SMART DEVICES A new investigation found that LG Smart TVs were reportedly scanning users’ home networks, tracking viewing activity, and recording nearby conversations, even when the television was disconnected from the internet. Thomas Drake warns that a TV screen can become a surveillance device. Anything connected to a network can potentially be exploited by private companies or the government. The concern goes beyond what you watch. Massive amounts of data are collected even when a device is no longer connected to the network. The modern TV is no longer just a screen in your living room. It functions as a network sensor, and potentially a listening device, inside one of the most constitutionally protected spaces in America: your home.

NSA WHISTLEBLOWER REACTS TO LG TV SPYWARE INVESTIGATION, WARNS ABOUT SMART DEVICES A new investigation found that LG Smart TVs were reportedly scanning users’ home networks, tracking viewing activity, and recording nearby conversations, even when the television was disconnected from the internet. Thomas Drake warns that a TV screen can become a surveillance device. Anything connected to a network can potentially be exploited by private companies or the government. The concern goes beyond what you watch. Massive amounts of data are collected even when a device is no longer connected to the network. The modern TV is no longer just a screen in your living room. It functions as a network sensor, and potentially a listening device, inside one of the most constitutionally protected spaces in America: your home.

Redacted

17,236 Aufrufe • vor 22 Tagen

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,756 Aufrufe • vor 2 Jahren

IN 1986 MIT FILMED THE LECTURE WHERE CODE STOPPED BEING CODE AND BECAME DATA 43 minutes from Gerald Sussman, in the most legendary intro programming course ever recorded. -> The idea that lands: to a program, your code is just data it can read and rewrite. He builds a program that does real calculus -- not by crunching numbers, but by reading the math as a list and transforming it piece by piece. Then he shows the trick hiding under all of it: in this language, code and data are the same material. Forty years later that is exactly what an AI coding agent is -- a program that reads your code as data, rewrites it, and hands it back. Sussman drew the whole idea on a blackboard in 1986. Writing code was never the deepest skill -- understanding that code itself is data a program can manipulate is. This is where you learn it. Most people think AI writing code is brand new. The ones who watch this saw the blueprint 40 years ago. Bookmark & Watch it. This one's a legend ↓

slash1s

22,107 Aufrufe • vor 3 Monaten

THIS IS STRAIGHT UP THE CREEPIEST THING I'VE EVER SEEN they wired 166,700 reconstructed fruit fly neurons into this android just to see if it could move, and then it started doing things nobody programmed at first it was basic stuff. camera sees motion → fly network reacts → motors move. nothing special then it started turning toward people before they even reached it the team figured it was a bad sensor, so they stripped out every scripted behavior and ran the body again with only the fly network connected it did it again head tracks a person across the room hand closes when someone leans in too close body flinches when the lights go out the strangest part came later they left it alone in the room for 17 minutes and the activity never settled. the fly network kept firing with almost nothing coming in, like it was working on something by itself 166,700 neurons is nothing compared to a human brain but nobody told those neurons they'd ended up in a human-shaped body the whole point was to see if a fly brain could drive an android now they're trying to work out why it keeps looking back

THIS IS STRAIGHT UP THE CREEPIEST THING I'VE EVER SEEN they wired 166,700 reconstructed fruit fly neurons into this android just to see if it could move, and then it started doing things nobody programmed at first it was basic stuff. camera sees motion → fly network reacts → motors move. nothing special then it started turning toward people before they even reached it the team figured it was a bad sensor, so they stripped out every scripted behavior and ran the body again with only the fly network connected it did it again head tracks a person across the room hand closes when someone leans in too close body flinches when the lights go out the strangest part came later they left it alone in the room for 17 minutes and the activity never settled. the fly network kept firing with almost nothing coming in, like it was working on something by itself 166,700 neurons is nothing compared to a human brain but nobody told those neurons they'd ended up in a human-shaped body the whole point was to see if a fly brain could drive an android now they're trying to work out why it keeps looking back

explos1ve

96,910 Aufrufe • vor 3 Tagen