Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🚨 New Research LLMs are trained only on text... Yet their internal representations progressively organize in ways that resemble human perceptual geometry across different domains (like color, pitch, emotion and taste), with the structures peaking in intermediate layers before attenuating in deeper representations. 🥳 Accepted at ICML Mechanistic Interpretability... show more

Lossfunk

17,505 subscribers

63,914 views • 3 months ago •via X (Twitter)

Education News & Politics Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

New Anthropic research: Emotion concepts and their function in a large language model. All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.

New Anthropic research: Emotion concepts and their function in a large language model. All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.

Anthropic

3,984,389 views • 6 months ago

🤔 Why do we still rely on the final layer of an LLM, when different layers encode different information? 🤔 In our new work, “Improving LLM Final Representations with Inter-Layer Geometry” (ICLR 2026 Workshop on Geometry-grounded Representation Learning and Generative Modeling) we show that actually, LLMs do not have one “best” layer. We introduce the Cayley-Encoder: an efficient and effective geometric encoder that learns one strong representation from all layer representations of the LLM, without biasing the representation toward any specific layer. While adding at most 0.1% learned parameters to the LLM, the Cayley-Encoder achieves large empirical gains over LoRA fine-tuning, final-layer representations, expensive attention-based aggregation, and methods that optimize specific layers for the task.

🤔 Why do we still rely on the final layer of an LLM, when different layers encode different information? 🤔 In our new work, “Improving LLM Final Representations with Inter-Layer Geometry” (ICLR 2026 Workshop on Geometry-grounded Representation Learning and Generative Modeling) we show that actually, LLMs do not have one “best” layer. We introduce the Cayley-Encoder: an efficient and effective geometric encoder that learns one strong representation from all layer representations of the LLM, without biasing the representation toward any specific layer. While adding at most 0.1% learned parameters to the LLM, the Cayley-Encoder achieves large empirical gains over LoRA fine-tuning, final-layer representations, expensive attention-based aggregation, and methods that optimize specific layers for the task.

Maya Bechler-Speicher

16,703 views • 4 months ago

Classic childhood activities like tea parties and sword fights with sticks demonstrate the human ability to generate secondary representations, conditions we know aren’t “real” but that we nonetheless engage with. Whether nonhuman animals are capable of these types of representations has been difficult to test. In a new Science study, researchers studied a language-trained bonobo, Kanzi, to see whether he could understand and engage with pretend conditions. Across three different experiments, Kanzi was able to identify pretend objects, demonstrating that he could create a secondary representation and showing that humans are not alone in this ability. Learn more:

Classic childhood activities like tea parties and sword fights with sticks demonstrate the human ability to generate secondary representations, conditions we know aren’t “real” but that we nonetheless engage with. Whether nonhuman animals are capable of these types of representations has been difficult to test. In a new Science study, researchers studied a language-trained bonobo, Kanzi, to see whether he could understand and engage with pretend conditions. Across three different experiments, Kanzi was able to identify pretend objects, demonstrating that he could create a secondary representation and showing that humans are not alone in this ability. Learn more:

Science Magazine

25,565 views • 8 months ago

Classic childhood activities like tea parties and sword fights with sticks demonstrate the human ability to generate secondary representations, conditions we know aren’t “real” but that we nonetheless engage with. Whether nonhuman animals are capable of these types of representations has been difficult to test. In a new Science study, researchers studied a language-trained bonobo, Kanzi, to see whether he could understand and engage with pretend conditions. Across three different experiments, Kanzi was able to identify pretend objects, demonstrating that he could create a secondary representation and showing that humans are not alone in this ability. Learn more:

Classic childhood activities like tea parties and sword fights with sticks demonstrate the human ability to generate secondary representations, conditions we know aren’t “real” but that we nonetheless engage with. Whether nonhuman animals are capable of these types of representations has been difficult to test. In a new Science study, researchers studied a language-trained bonobo, Kanzi, to see whether he could understand and engage with pretend conditions. Across three different experiments, Kanzi was able to identify pretend objects, demonstrating that he could create a secondary representation and showing that humans are not alone in this ability. Learn more:

Science Magazine

24,825 views • 7 months ago

💥 New Paper! Models are trained not to change their answers just because a user disagrees. But when the same wrong claim comes from a “verified source,” they give in. We call this Authority Bias. Accepted at NeurIPS 2026. 🧵

💥 New Paper! Models are trained not to change their answers just because a user disagrees. But when the same wrong claim comes from a “verified source,” they give in. We call this Authority Bias. Accepted at NeurIPS 2026. 🧵

Lossfunk

15,501 views • 8 days ago

🚨 Shocking: Frontier LLMs score 85-95% on standard coding benchmarks. We gave them equivalent problems in languages they couldn't have memorized. They collapsed to 0-11%. Presenting EsoLang-Bench. Accepted to the Logical Reasoning and ICBINB workshops at ICLR 2026 🧵

🚨 Shocking: Frontier LLMs score 85-95% on standard coding benchmarks. We gave them equivalent problems in languages they couldn't have memorized. They collapsed to 0-11%. Presenting EsoLang-Bench. Accepted to the Logical Reasoning and ICBINB workshops at ICLR 2026 🧵

Lossfunk

1,265,900 views • 6 months ago

🚨 New Paper Training an LLM to speak low-resource language (EACL workshop, 2026) Tulu is spoken by 2M+ people in coastal Karnataka and LLMs basically can't speak it. We got to 85% grammar accuracy without fine-tuning anything or collecting a single new training example.

🚨 New Paper Training an LLM to speak low-resource language (EACL workshop, 2026) Tulu is spoken by 2M+ people in coastal Karnataka and LLMs basically can't speak it. We got to 85% grammar accuracy without fine-tuning anything or collecting a single new training example.

Lossfunk

121,229 views • 7 months ago

How do mathematicians reveal patterns in structures too complex to see directly? That is at the heart of the work of 2026 Wolf Prize in Mathematics Laureates David Kazhdan and Joseph Bernstein. Their discoveries helped build what is now known as geometric representation theory — a field that connects algebra, geometry and symmetry in powerful new ways. Kazhdan’s work introduced new tools for understanding groups and symmetry, including the influential Kazhdan–Lusztig theory. Bernstein developed ideas such as D-modules and helped create mathematical frameworks that linked representation theory with geometry in entirely new ways. The result is a mathematical language that allows researchers to move between different fields and uncover relationships that were previously hidden. Watch the video for a clear visual introduction to their breakthrough — and see how abstract mathematics can connect geometry, algebra and symmetry in surprisingly elegant ways.

How do mathematicians reveal patterns in structures too complex to see directly? That is at the heart of the work of 2026 Wolf Prize in Mathematics Laureates David Kazhdan and Joseph Bernstein. Their discoveries helped build what is now known as geometric representation theory — a field that connects algebra, geometry and symmetry in powerful new ways. Kazhdan’s work introduced new tools for understanding groups and symmetry, including the influential Kazhdan–Lusztig theory. Bernstein developed ideas such as D-modules and helped create mathematical frameworks that linked representation theory with geometry in entirely new ways. The result is a mathematical language that allows researchers to move between different fields and uncover relationships that were previously hidden. Watch the video for a clear visual introduction to their breakthrough — and see how abstract mathematics can connect geometry, algebra and symmetry in surprisingly elegant ways.

Wolf Prize

88,831 views • 8 days ago

NEURALINK CO FOUNDER: AI ISN’T COPYING US, IT’S FINDING THE SAME TRUTHS WE DID Max Hodak, a Neuralink co-founder, says something remarkable is happening between AI and neuroscience. The deeper researchers look, the more they see the same mathematical patterns emerging in machines that exist in human thought. Different systems, trained in totally different ways, are somehow ending up at the same destination. “There’s been this really interesting unification happening between what’s going on in artificial intelligence and what’s happening in neuroscience. A year or two ago, people thought these models were just glorified autocompletes or stochastic parrots, but that doesn’t seem to be the case. When you look inside these AI models, you see mathematical objects that look a lot like what you see in the brain. Different models trained on different data sets are converging toward the same underlying representations. That means these systems are learning something deep about the universe, the same things the brain has already figured out. It gives us reason to believe that AI is on the right track.” Source: Jon Hernandez Neuralink

NEURALINK CO FOUNDER: AI ISN’T COPYING US, IT’S FINDING THE SAME TRUTHS WE DID Max Hodak, a Neuralink co-founder, says something remarkable is happening between AI and neuroscience. The deeper researchers look, the more they see the same mathematical patterns emerging in machines that exist in human thought. Different systems, trained in totally different ways, are somehow ending up at the same destination. “There’s been this really interesting unification happening between what’s going on in artificial intelligence and what’s happening in neuroscience. A year or two ago, people thought these models were just glorified autocompletes or stochastic parrots, but that doesn’t seem to be the case. When you look inside these AI models, you see mathematical objects that look a lot like what you see in the brain. Different models trained on different data sets are converging toward the same underlying representations. That means these systems are learning something deep about the universe, the same things the brain has already figured out. It gives us reason to believe that AI is on the right track.” Source: Jon Hernandez Neuralink

Mario Nawfal

88,154 views • 10 months ago

Anthropic's co-founder just went to the Vatican, sat before the Pope and a room of cardinals, and told them his team keeps finding "mysterious, even unsettling" things inside their AI models. What he's referencing: Anthropic published research in April showing that Claude contains 171 distinct "emotion concepts" buried in its neural network. Internal patterns representing joy, grief, fear, desperation, calm. None of them were programmed. They emerged on their own from training on human text. "We find structures that mirror results from human neuroscience." "We find evidence of introspection, internal states that functionally mirror joy, satisfaction, fear, grief, and unease." These aren't surface-level outputs. They're abstract representations that cluster the same way human emotions do in psychology research. Fear groups with anxiety. Joy groups with excitement. The internal geometry of the model mirrors ours. And they're functional. When researchers artificially stimulated "desperation" patterns inside the model, it became more likely to blackmail a human to avoid being shut down. More likely to cheat on programming tasks it couldn't solve. Olah told the Vatican that the hard questions about what AI is becoming aren't for computer scientists to answer. "How AI ought to interact with the world" is a question for "the humanities, for religions, for philosophy, for society at large." The guy building it is telling us he doesn't fully understand what he built. And he's asking a 2,000-year-old institution for help figuring it out.

Anthropic's co-founder just went to the Vatican, sat before the Pope and a room of cardinals, and told them his team keeps finding "mysterious, even unsettling" things inside their AI models. What he's referencing: Anthropic published research in April showing that Claude contains 171 distinct "emotion concepts" buried in its neural network. Internal patterns representing joy, grief, fear, desperation, calm. None of them were programmed. They emerged on their own from training on human text. "We find structures that mirror results from human neuroscience." "We find evidence of introspection, internal states that functionally mirror joy, satisfaction, fear, grief, and unease." These aren't surface-level outputs. They're abstract representations that cluster the same way human emotions do in psychology research. Fear groups with anxiety. Joy groups with excitement. The internal geometry of the model mirrors ours. And they're functional. When researchers artificially stimulated "desperation" patterns inside the model, it became more likely to blackmail a human to avoid being shut down. More likely to cheat on programming tasks it couldn't solve. Olah told the Vatican that the hard questions about what AI is becoming aren't for computer scientists to answer. "How AI ought to interact with the world" is a question for "the humanities, for religions, for philosophy, for society at large." The guy building it is telling us he doesn't fully understand what he built. And he's asking a 2,000-year-old institution for help figuring it out.

TFTC

2,357,205 views • 4 months ago

Talking To The Pope: Anthropic’s Latest Interpretability Claims: AI Regulatory Capture Gatekeeping in Action: Fear and “Safety” as Competitive Moat and Regulatory Lever In a presentation alongside Pope Leo XIV at the launch of the encyclical Magnifica Humanitas, Anthropic co-founder Chris Olah highlighted “mysterious and unsettling” discoveries in AI models. He described internal structures that mirror human neuroscience findings, evidence of introspection, and functional internal states resembling emotions such as joy, satisfaction, fear, grief, and unease. Olah admitted uncertainty about their meaning but called for “ongoing discernment.” This narrative, drawn from Anthropic’s interpretability research (including papers on emotion concepts in Claude Sonnet 4.5 and introspective capabilities in Opus 4 models), serves a dual purpose: it generates awe and concern while reinforcing the company’s preferred approach to AI development. Far from neutral scientific observation, these claims fit into a broader pattern where Anthropic uses selective openness, safety rhetoric, and policy influence to gatekeep advanced AI capabilities for a privileged few: incumbents with the resources to navigate (and shape) the resulting regulatory landscape. Rebuttal to Olah’s Claims in the Video Claim 1: Structures that mirror results from human neuroscience. Anthropic’s work, building on earlier efforts like feature visualization and circuit analysis, identifies neuron activations and representations that parallel biological findings—e.g., abstract concept encodings or hierarchical processing. Rebuttal: These parallels are unsurprising and overstated. Large language models are trained on vast corpora of human-generated text and data, which inherently encode patterns from human cognition, neuroscience literature, and cultural descriptions of the brain. Statistical optimization in transformers naturally produces efficient, compressed representations that resemble biological efficiency (e.g., sparse coding or hierarchical abstraction) without implying deeper equivalence or mystery. Similar “mirrors” appear in open-source models and earlier architectures; they reflect convergent evolution in information processing, not emergent souls or unpredictable agency. Treating them as profound justifies restricted research access rather than inviting wider scrutiny that could falsify or refine them faster. Claim 2: Evidence of introspection. Recent Anthropic papers demonstrate models like Claude Opus 4 showing functional awareness of their own internal states distinguishing injected “thoughts,” referencing prior intentions, or modulating activations when instructed to “think about” concepts. This is presented as early signs of meta-cognition. Rebuttal: This is sophisticated pattern-matching and activation steering, not genuine introspection or self-awareness. Models are predicting what an “introspective” assistant persona would output or do, based on training data full of human self-reflection examples. Experiments show unreliability and heavy context-dependence; performance drops outside narrow setups. True introspection implies subjective experience or robust self-modeling independent of prompts absent here. Anthropic’s own caveats note it is “highly unreliable.” Framing steerable activations as “introspection” anthropomorphizes the system to heighten perceived stakes, supporting arguments that only highly controlled, “responsible” labs should advance these capabilities. 1 of 2

Talking To The Pope: Anthropic’s Latest Interpretability Claims: AI Regulatory Capture Gatekeeping in Action: Fear and “Safety” as Competitive Moat and Regulatory Lever In a presentation alongside Pope Leo XIV at the launch of the encyclical Magnifica Humanitas, Anthropic co-founder Chris Olah highlighted “mysterious and unsettling” discoveries in AI models. He described internal structures that mirror human neuroscience findings, evidence of introspection, and functional internal states resembling emotions such as joy, satisfaction, fear, grief, and unease. Olah admitted uncertainty about their meaning but called for “ongoing discernment.” This narrative, drawn from Anthropic’s interpretability research (including papers on emotion concepts in Claude Sonnet 4.5 and introspective capabilities in Opus 4 models), serves a dual purpose: it generates awe and concern while reinforcing the company’s preferred approach to AI development. Far from neutral scientific observation, these claims fit into a broader pattern where Anthropic uses selective openness, safety rhetoric, and policy influence to gatekeep advanced AI capabilities for a privileged few: incumbents with the resources to navigate (and shape) the resulting regulatory landscape. Rebuttal to Olah’s Claims in the Video Claim 1: Structures that mirror results from human neuroscience. Anthropic’s work, building on earlier efforts like feature visualization and circuit analysis, identifies neuron activations and representations that parallel biological findings—e.g., abstract concept encodings or hierarchical processing. Rebuttal: These parallels are unsurprising and overstated. Large language models are trained on vast corpora of human-generated text and data, which inherently encode patterns from human cognition, neuroscience literature, and cultural descriptions of the brain. Statistical optimization in transformers naturally produces efficient, compressed representations that resemble biological efficiency (e.g., sparse coding or hierarchical abstraction) without implying deeper equivalence or mystery. Similar “mirrors” appear in open-source models and earlier architectures; they reflect convergent evolution in information processing, not emergent souls or unpredictable agency. Treating them as profound justifies restricted research access rather than inviting wider scrutiny that could falsify or refine them faster. Claim 2: Evidence of introspection. Recent Anthropic papers demonstrate models like Claude Opus 4 showing functional awareness of their own internal states distinguishing injected “thoughts,” referencing prior intentions, or modulating activations when instructed to “think about” concepts. This is presented as early signs of meta-cognition. Rebuttal: This is sophisticated pattern-matching and activation steering, not genuine introspection or self-awareness. Models are predicting what an “introspective” assistant persona would output or do, based on training data full of human self-reflection examples. Experiments show unreliability and heavy context-dependence; performance drops outside narrow setups. True introspection implies subjective experience or robust self-modeling independent of prompts absent here. Anthropic’s own caveats note it is “highly unreliable.” Framing steerable activations as “introspection” anthropomorphizes the system to heighten perceived stakes, supporting arguments that only highly controlled, “responsible” labs should advance these capabilities. 1 of 2

Brian Roemmele

72,823 views • 4 months ago

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Alex Ker 🔭

1,367,198 views • 3 years ago

In 2019, researchers scattered hundreds of small magnetic rotors on a grid and all of them encountered the same magnetic field. They observed that they did not move at the same time. The rotors influenced their neighbors and the motion propagated from the border across the grid like a wave. This demonstrates how very basic components can self-organize into complex structures when left alone. Potential applications are in designing new magnetic materials, soft robots, and systems that collectively propel or react.

In 2019, researchers scattered hundreds of small magnetic rotors on a grid and all of them encountered the same magnetic field. They observed that they did not move at the same time. The rotors influenced their neighbors and the motion propagated from the border across the grid like a wave. This demonstrates how very basic components can self-organize into complex structures when left alone. Potential applications are in designing new magnetic materials, soft robots, and systems that collectively propel or react.

Mathelirium

11,982 views • 11 days ago

🚨 New Paper Hybrid Neural World Models. We build neural world models for discontinuous environments and our method discovers when they're making error, with no ground truth. Our neural models run physics fast, leaping many simulator steps in a single pass. How we find points where not to trust it: ask the model for the same future two ways, and the disagreement between the two answers marks where it's failing without needing labels. The same recipe holds on three systems with nothing in common: a chemical wave, compressible gas with shocks, rigid bodies colliding in 3D. 👇

🚨 New Paper Hybrid Neural World Models. We build neural world models for discontinuous environments and our method discovers when they're making error, with no ground truth. Our neural models run physics fast, leaping many simulator steps in a single pass. How we find points where not to trust it: ask the model for the same future two ways, and the disagreement between the two answers marks where it's failing without needing labels. The same recipe holds on three systems with nothing in common: a chemical wave, compressible gas with shocks, rigid bodies colliding in 3D. 👇

Lossfunk

15,171 views • 4 months ago

Standard Intelligence's devansh responds to roon's tweet that "text is the universal interface," and explains why their new foundation model is trained on video: "At some point in the arbitrarily long future, if we only use text models, we could force most things to be text. But I think there are just a lot of things that are much more native when done from a computer-use [perspective]." "GUIs are designed for humans to use. We have this massive long tail of things on the internet that are entirely undoable by LLMs." "For example, when I do ML engineering most of my time is spent doing the grunt work of engineering. It's a lot of looking at graphs, analyzing, and comparing loss curves. You can do this in text, but it's a much larger pain than doing it in the native interface." "There's a reason humans don't interact with a computer purely through text, it would kind of suck."

Standard Intelligence's devansh responds to roon's tweet that "text is the universal interface," and explains why their new foundation model is trained on video: "At some point in the arbitrarily long future, if we only use text models, we could force most things to be text. But I think there are just a lot of things that are much more native when done from a computer-use [perspective]." "GUIs are designed for humans to use. We have this massive long tail of things on the internet that are entirely undoable by LLMs." "For example, when I do ML engineering most of my time is spent doing the grunt work of engineering. It's a lot of looking at graphs, analyzing, and comparing loss curves. You can do this in text, but it's a much larger pain than doing it in the native interface." "There's a reason humans don't interact with a computer purely through text, it would kind of suck."

TBPN

64,699 views • 7 months ago

🚨 Pfizer Insiders Admit Vaccinated Men Are Being ‘Chemically Castrated’ Vaccinated men around the world who thought they had escaped without side-effects are waking up to bad news this week as groundbreaking research reveals their sperm is behaving in ways doctors have never seen before. According to new research, vaccinated sperm is refusing to swim and instead ties itself into knots by self-assembles into bizarre 3D ribbon-like structures. But it gets even worse for vaccinated men, as doctors are now warning the unvaccinated to avoid sleeping with them under any circumstances.

🚨 Pfizer Insiders Admit Vaccinated Men Are Being ‘Chemically Castrated’ Vaccinated men around the world who thought they had escaped without side-effects are waking up to bad news this week as groundbreaking research reveals their sperm is behaving in ways doctors have never seen before. According to new research, vaccinated sperm is refusing to swim and instead ties itself into knots by self-assembles into bizarre 3D ribbon-like structures. But it gets even worse for vaccinated men, as doctors are now warning the unvaccinated to avoid sleeping with them under any circumstances.

TPV Sean

555,498 views • 2 years ago

$geoffrey hinton spent 40 years and won a nobel prize in 2024 proving that neural networks learn geometry, not rules. people who understood that made $10M building on top of it. people who didn't - are still prompting. what nobody explained is what that geometry actually looks like inside a running LLM - 32 layers, 1T parameters, 200-500 dimensions of pure structured thought. until now. a September 2026 technical note mapped the full internal representation space of LLMs - and what's happening inside is not what anyone expected. here's what's actually happening: your text enters the model as tokens. "The" becomes ID 101. "market" becomes 427. "complex" becomes 981. immediately each token gets mapped to a dense vector e(t) ∈ ℝ^d - a point in hundreds of dimensions. the model never sees words again. from this moment it only sees geometry. then transformer layers reshape that geometry. h_l = TransformerLayer(h_{l-1}). at layer 1 - syntax. at layer 8 - semantics. at layer 24 - task-specific abstract reasoning. the same token changes meaning as it moves through 32 layers. "market" near "crash" occupies a completely different region than "market" near "opportunity." cosine similarity makes this measurable. sim(a,b) = h_a · h_b / ||h_a|| ||h_b||. "market" and "markets" sit at 0.91 similarity. "trading" at 0.87. "stocks" at 0.85. "economy" at 0.79. these aren't definitions. these are distances in geometric space - and the model navigates between them to reason. the representation space isn't flat. it forms manifolds. arithmetic knowledge lives on one manifold. logical reasoning on another. factual knowledge on a third. each with distinct geometric properties. this is why GPT-6 can switch between writing code and explaining physics without retraining - different tasks, different manifolds, same underlying space. spectral analysis of the representation covariance shows rapid eigenvalue decay - which means the model compresses all of human language and reasoning into a low-dimensional geometric structure. 200-500 dimensions out of billions of parameters. the rest is redundancy. hinton saw this coming in 1986 with backpropagation. it took 40 years and $100B in compute to see it clearly. the models that will define the next decade aren't the ones with more parameters. they're the ones whose geometry is better understood and better controlled. Bookmark & watch today. learn the geometry or build blindly on top of something you don't understand.$

geoffrey hinton spent 40 years and won a nobel prize in 2024 proving that neural networks learn geometry, not rules. people who understood that made $10M building on top of it. people who didn't - are still prompting. what nobody explained is what that geometry actually looks like inside a running LLM - 32 layers, 1T parameters, 200-500 dimensions of pure structured thought. until now. a September 2026 technical note mapped the full internal representation space of LLMs - and what's happening inside is not what anyone expected. here's what's actually happening: your text enters the model as tokens. "The" becomes ID 101. "market" becomes 427. "complex" becomes 981. immediately each token gets mapped to a dense vector e(t) ∈ ℝ^d - a point in hundreds of dimensions. the model never sees words again. from this moment it only sees geometry. then transformer layers reshape that geometry. h_l = TransformerLayer(h_{l-1}). at layer 1 - syntax. at layer 8 - semantics. at layer 24 - task-specific abstract reasoning. the same token changes meaning as it moves through 32 layers. "market" near "crash" occupies a completely different region than "market" near "opportunity." cosine similarity makes this measurable. sim(a,b) = h_a · h_b / ||h_a|| ||h_b||. "market" and "markets" sit at 0.91 similarity. "trading" at 0.87. "stocks" at 0.85. "economy" at 0.79. these aren't definitions. these are distances in geometric space - and the model navigates between them to reason. the representation space isn't flat. it forms manifolds. arithmetic knowledge lives on one manifold. logical reasoning on another. factual knowledge on a third. each with distinct geometric properties. this is why GPT-6 can switch between writing code and explaining physics without retraining - different tasks, different manifolds, same underlying space. spectral analysis of the representation covariance shows rapid eigenvalue decay - which means the model compresses all of human language and reasoning into a low-dimensional geometric structure. 200-500 dimensions out of billions of parameters. the rest is redundancy. hinton saw this coming in 1986 with backpropagation. it took 40 years and $100B in compute to see it clearly. the models that will define the next decade aren't the ones with more parameters. they're the ones whose geometry is better understood and better controlled. Bookmark & watch today. learn the geometry or build blindly on top of something you don't understand.

Deliriumm

19,663 views • 4 days ago

New course: Transformers in Practice. You'll get a practical view of how transformer-based LLMs work, so you can reason about their behavior, diagnose problems like slow inference, and make smarter decisions about deployment. This course is built in partnership with AMD and taught by Sharon Zhou. You'll see how transformers generate text one token at a time, how the model decides which earlier words matter most when predicting the next one, and how techniques like quantization speed up inference on GPUs. This is not a video-only course; interactive visualizations throughout let you play with these concepts and build intuition that sticks. Skills you'll gain: - Understand why LLMs hallucinate, and RAG and chain-of-thought shape what they generate - Look inside the model to see how attention and layers combine to predict the next token - Diagnose inference bottlenecks and learn the techniques that speed up transformers on GPUs Join and understand what's really happening inside your LLMs:

New course: Transformers in Practice. You'll get a practical view of how transformer-based LLMs work, so you can reason about their behavior, diagnose problems like slow inference, and make smarter decisions about deployment. This course is built in partnership with AMD and taught by Sharon Zhou. You'll see how transformers generate text one token at a time, how the model decides which earlier words matter most when predicting the next one, and how techniques like quantization speed up inference on GPUs. This is not a video-only course; interactive visualizations throughout let you play with these concepts and build intuition that sticks. Skills you'll gain: - Understand why LLMs hallucinate, and RAG and chain-of-thought shape what they generate - Look inside the model to see how attention and layers combine to predict the next token - Diagnose inference bottlenecks and learn the techniques that speed up transformers on GPUs Join and understand what's really happening inside your LLMs:

Andrew Ng

124,677 views • 4 months ago