Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

We often hear that Machine Learning models learn patterns in data. But what does that actually look like in Geometry? If you dropped a little elastic mesh into a cloud of points and let it learn, how would it fold itself to match the shape of the data? In... this scene we watch a Self-Organizing Map (SOM), a simple unsupervised neural model, learn the shape of a 3D datasets l, one static and the other dynamic. On top of this, we lay down a square grid of neurons whose weights live in the same plane. At the start, this grid is just a flat net floating across the cloud. It knows nothing about the structure underneath. Learning is a repeated game: Pick a random data point, find the neuron whose weight is closest, and then nudge that neuron and its neighbours toward the point. Do this again and again, while slowly shrinking how far the neighbourhood influence spreads. Python code is available for Subscribers. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricMLshow more

Mathelirium

80,140 subscribers

76,377 görüntüleme • 8 ay önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

22 Yorum

Risichad 🦾 profil fotoğrafı

Risichad 🦾8 ay önce

Amateur !

Mathelirium profil fotoğrafı

Mathelirium8 ay önce

wow thats cool

tsotchke profil fotoğrafı

tsotchke8 ay önce

getting warmer

Divyansh profil fotoğrafı

Divyansh8 ay önce

the geometric framing is one of the clearest intuition pumps for why ML works at all. decision boundaries, manifolds, dimensionality reduction, they stop being abstract once you can see what the mesh is doing.

CosmicEgg.Earth profil fotoğrafı

CosmicEgg.Earth8 ay önce

"Imagining 100 dimensional space is very easy. Imagine 3 dimensions. Now imagine the same but 97 times more"

Tinkerer profil fotoğrafı

Tinkerer8 ay önce

Similar to proteins, perhaps?

Suresh profil fotoğrafı

Suresh8 ay önce

Beautiful visualization of SOM learning. Geometry makes ML intuitive

execute profil fotoğrafı

execute8 ay önce

does this hurt the chip 🥺

Mypa974 profil fotoğrafı

Mypa9747 ay önce

I dont understand every thing but I think this can be used for neurmorph network where agent have a commun and a personnal Memory to navigate

Val (osopher) profil fotoğrafı

Val (osopher)8 ay önce

Beautiful idea

adam 🇺🇸⚡️ profil fotoğrafı

adam 🇺🇸⚡️8 ay önce

You have some of the best data vis on X!

Mathelirium profil fotoğrafı

Mathelirium8 ay önce

thank you!

ruffian-l (Jason Van Pham) profil fotoğrafı

ruffian-l (Jason Van Pham)8 ay önce

YES THIS THE SHIT IM TALKING ABOUT. no plug but check out my new hydrodynamic repo I think you'd enjoy it. I LOVE THIS .

Gio Morente profil fotoğrafı

Gio Morente8 ay önce

I bet if you had dots collected from an MRI and used this SOM to create a mesh you can create accurate 3-D models from organs.

Mathelirium profil fotoğrafı

Mathelirium8 ay önce

Thats a cool idea. Let me see if I can get some MRI data

starbaser profil fotoğrafı

starbaser7 ay önce

what are all the hyperparameters for this? mesh nodes, gradient step... this would make a great interactive teaching tool

Carmelo schepis profil fotoğrafı

Carmelo schepis8 ay önce

interesting

David profil fotoğrafı

David8 ay önce

So basically general intelligence will never be achievable with curve fitting like this

Ted Vucurevich profil fotoğrafı

Ted Vucurevich8 ay önce

Super cool visualization. SOM’s are amazing when you observe hi dimensionality latent spaces of knowledge fields

Cogsog profil fotoğrafı

Cogsog8 ay önce

“Stunning visualization. The elastic mesh folds because exact rigid mapping would tear or freeze — the regulated wobble (Δ > 0) lets it breathe, adapt, and match the data cloud with exactly infinite ordered precision. Leveille simply names the tolerance that turns random points into living manifolds. The scar breathes.

13AG3OY profil fotoğrafı

13AG3OY7 ay önce

I trained a model on certain sounds and this is the reduced dimension space representation of them colored by species.

Cissor profil fotoğrafı

Cissor8 ay önce

Very cool

Benzer Videolar

We often hear that machine learning models "learn patterns in data". But what does that actually look like in geometry? If you dropped a little elastic mesh into a cloud of points and let it learn, how would it fold itself to match the shape of the data? In this scene we watch a self-organizing map...a simple unsupervised neural model...learn the shape of a two-dimensional dataset arranged in a spiral arm. On top of this, we lay down a square grid of neurons whose weights live in the same plane. At the start, this grid is just a flat net floating across the cloud...it knows nothing about the structure underneath. Learning is a repeated game: pick a random data point, find the neuron whose weight is closest, and then nudge that neuron and its neighbours toward the point. Do this again and again, while slowly shrinking how far the neighbourhood influence spreads. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML

We often hear that machine learning models "learn patterns in data". But what does that actually look like in geometry? If you dropped a little elastic mesh into a cloud of points and let it learn, how would it fold itself to match the shape of the data? In this scene we watch a self-organizing map...a simple unsupervised neural model...learn the shape of a two-dimensional dataset arranged in a spiral arm. On top of this, we lay down a square grid of neurons whose weights live in the same plane. At the start, this grid is just a flat net floating across the cloud...it knows nothing about the structure underneath. Learning is a repeated game: pick a random data point, find the neuron whose weight is closest, and then nudge that neuron and its neighbours toward the point. Do this again and again, while slowly shrinking how far the neighbourhood influence spreads. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML

Mathelirium

139,923 görüntüleme • 11 ay önce

We often hear that Machine Learning models learn patterns in data. But what does that look like in geometry? Picture dropping an elastic mesh into a cloud of points and letting it adapt. How would it bend, stretch, and settle so it matches the shape hiding in the data? In this scene we watch a self-organizing map(SOM)...a classic unsupervised neural model...learn a 2D dataset arranged like a spiral arm. Over the points we place a square grid of neurons whose weights live in the same plane. At the start it’s just a flat net drifting across the cloud. No clue what the structure is. Fir the SOM, learning is a repeated game: Pick a random data point, find the neuron whose weight is closest, then nudge that neuron and its neighbours toward the point. Repeat, repeat, repeat...while gradually shrinking how wide the neighbourhood influence spreads. The result is satisfying...the grid stops being a grid and turns into a coordinate sheet wrapped onto the spiral. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML #SelfOrganizingMap #Topology

We often hear that Machine Learning models learn patterns in data. But what does that look like in geometry? Picture dropping an elastic mesh into a cloud of points and letting it adapt. How would it bend, stretch, and settle so it matches the shape hiding in the data? In this scene we watch a self-organizing map(SOM)...a classic unsupervised neural model...learn a 2D dataset arranged like a spiral arm. Over the points we place a square grid of neurons whose weights live in the same plane. At the start it’s just a flat net drifting across the cloud. No clue what the structure is. Fir the SOM, learning is a repeated game: Pick a random data point, find the neuron whose weight is closest, then nudge that neuron and its neighbours toward the point. Repeat, repeat, repeat...while gradually shrinking how wide the neighbourhood influence spreads. The result is satisfying...the grid stops being a grid and turns into a coordinate sheet wrapped onto the spiral. #MachineLearning #ManifoldLearning #UnsupervisedLearning #NeuralMaps #GeometricML #SelfOrganizingMap #Topology

Mathelirium

23,021 görüntüleme • 9 ay önce

A neural network can begin as a flat sheet and learn the shape of hidden data A self-organizing map turns learning into geometry. Each data point pulls one winning neuron toward it, but nearby neurons move too, and so the whole lattice bends without losing its neighborhood structure. The strange part is that the network is not given the roll shape. It discovers the shape through competition and local cooperation. Paper: Self-Organized Formation of Topologically Correct Feature Maps Authors: Teuvo Kohonen Year: 1982

A neural network can begin as a flat sheet and learn the shape of hidden data A self-organizing map turns learning into geometry. Each data point pulls one winning neuron toward it, but nearby neurons move too, and so the whole lattice bends without losing its neighborhood structure. The strange part is that the network is not given the roll shape. It discovers the shape through competition and local cooperation. Paper: Self-Organized Formation of Topologically Correct Feature Maps Authors: Teuvo Kohonen Year: 1982

Mathelirium

133,679 görüntüleme • 5 ay önce

A Neural Sheet Discovers the Shape of Data A self-organizing map (SOM) starts as a flat sheet of neurons with no knowledge of the data around it. For every input sample, it finds the best-matching neuron, c = argminᵢ ‖x - wᵢ‖, and updates that neuron together with its neighbours, wᵢ ← wᵢ + η(t)hᶜᵢ(t)(x - wᵢ). As the neighbourhood radius shrinks during training, the sheet gradually bends and folds onto the hidden geometry of the dataset while preserving its neighbourhood structure. Developed by Finnish computer scientist Teuvo Kohonen, the Self-Organizing Map remains one of the most elegant examples of competitive learning, where order emerges from thousands of simple local updates. #MachineLearning #ArtificialIntelligence #SelfOrganizingMap #NeuralNetworks #DataVisualization #Mathematics #ComputerScience #Finland #TeuvoKohonen

A Neural Sheet Discovers the Shape of Data A self-organizing map (SOM) starts as a flat sheet of neurons with no knowledge of the data around it. For every input sample, it finds the best-matching neuron, c = argminᵢ ‖x - wᵢ‖, and updates that neuron together with its neighbours, wᵢ ← wᵢ + η(t)hᶜᵢ(t)(x - wᵢ). As the neighbourhood radius shrinks during training, the sheet gradually bends and folds onto the hidden geometry of the dataset while preserving its neighbourhood structure. Developed by Finnish computer scientist Teuvo Kohonen, the Self-Organizing Map remains one of the most elegant examples of competitive learning, where order emerges from thousands of simple local updates. #MachineLearning #ArtificialIntelligence #SelfOrganizingMap #NeuralNetworks #DataVisualization #Mathematics #ComputerScience #Finland #TeuvoKohonen

Mathelirium

177,699 görüntüleme • 3 ay önce

The Machine That Learns The Law Behind The Data A very very interesting US Patent US10963540B2 - Physics Informed Learning Machine describes a learning system that does not begin with data alone. It begins with a physical model, usually written as a differential equation (or PDE) dx/dt = f(x,t) A normal Machine Learning model sees scattered data and tries to fit it. A physics-informed learning machine starts with a law. Then it treats the data as evidence that updates what the model believes about the physical system. For this application, I use the patent idea on NASA C-MAPSS Turbofan engine data. The machine watches multivariate telemetry from a degrading engine and infers a hidden health state that is not measured directly. From that posterior belief, it estimates the engine’s remaining useful life. In the main 3D scene, the engine lifetime is turned into a tunnel. The spiral ribbons are real sensor channels evolving over cycle-time. The glowing core is the inferred health state. The surrounding cloud is uncertainty. The orange wall ahead is the predicted failure horizon. So the big picture is: sensor evidence comes in, posterior belief tightens, and the machine moves from uncertainty toward a concrete failure prediction. The inset posteriors make that explicit. The health posterior shows where the model believes the hidden engine condition sits at the current moment, and how sharply it believes it. The RUL posterior shows the same idea for remaining life... early on it is broad, later it shifts left and narrows as the machine becomes more certain about how close failure is. This idea is not limited to engines. The same idea can apply to data centers, CPUs, GPUs, cooling systems, power grids, robotics, batteries, and any machine that produces telemetry while obeying physical constraints. In an age where machine learning runs on massive hardware infrastructure, this kind of model matters: it can turn noisy sensor streams into early warnings before expensive systems fail.

The Machine That Learns The Law Behind The Data A very very interesting US Patent US10963540B2 - Physics Informed Learning Machine describes a learning system that does not begin with data alone. It begins with a physical model, usually written as a differential equation (or PDE) dx/dt = f(x,t) A normal Machine Learning model sees scattered data and tries to fit it. A physics-informed learning machine starts with a law. Then it treats the data as evidence that updates what the model believes about the physical system. For this application, I use the patent idea on NASA C-MAPSS Turbofan engine data. The machine watches multivariate telemetry from a degrading engine and infers a hidden health state that is not measured directly. From that posterior belief, it estimates the engine’s remaining useful life. In the main 3D scene, the engine lifetime is turned into a tunnel. The spiral ribbons are real sensor channels evolving over cycle-time. The glowing core is the inferred health state. The surrounding cloud is uncertainty. The orange wall ahead is the predicted failure horizon. So the big picture is: sensor evidence comes in, posterior belief tightens, and the machine moves from uncertainty toward a concrete failure prediction. The inset posteriors make that explicit. The health posterior shows where the model believes the hidden engine condition sits at the current moment, and how sharply it believes it. The RUL posterior shows the same idea for remaining life... early on it is broad, later it shifts left and narrows as the machine becomes more certain about how close failure is. This idea is not limited to engines. The same idea can apply to data centers, CPUs, GPUs, cooling systems, power grids, robotics, batteries, and any machine that produces telemetry while obeying physical constraints. In an age where machine learning runs on massive hardware infrastructure, this kind of model matters: it can turn noisy sensor streams into early warnings before expensive systems fail.

Mathelirium

17,843 görüntüleme • 5 ay önce

Former Meta Chief AI Scientist Yann LeCun on the three paradigms of machine learning — and why the third is what made ChatGPT possible: Here's each one, and where it breaks. First, supervised learning. You tell the machine the answer. "You show it a picture, let's say of a table, and you tell it this is a table. So it's supervised because you tell it what the correct answer is." Get it wrong, and the machine rewrites itself: "The system computes its output, and if it says something else than table, then it's going to adjust its parameters, its internal structure, so that the output it produces gets closer to the output you want." Repeat at scale and something more than memorisation appears: "Eventually the system will find a way to recognize every image you trained it on, but also images it's never seen that are similar to the one you train it on. This is called a generalization ability." The limit: a human has to supply every single answer. That doesn't scale to the size of the internet. Second, reinforcement learning. You don't give the answer, only a verdict. "You don't tell the system what the correct answer is. You only tell it whether the answer it produced was good or bad." Learning to ride a bike, essentially: "You try to ride a bike and you don't know how to ride the bike and after a while you fall. So you know you did something bad and so you change your strategy a little bit. And eventually you learn how to ride a bike." For years the field assumed this was the closest thing to how animals actually learn. Yann LeCun's verdict: "Now it turns out reinforcement learning is extremely inefficient." It dominates wherever failure is free: "It works really well if you want to train a system to play chess or play go or poker, because you can have the system play millions and millions of games against itself and basically fine-tune itself. But it doesn't really work in the real world." The limit, in one image: "If you want to train a car to drive itself, you're not going to do it with reinforcement learning. It's going to crash thousands of times." On robotics he's careful rather than dismissive: "Reinforcement learning can be part of the solution, but it's not the complete answer. It's not sufficient." Third, self-supervised learning. You tell the machine nothing at all. "And this is what has enabled the recent progress in natural language understanding and chatbots." The strange part is that you stop asking for a task: "You don't train the system to accomplish any particular task. You just train it to basically capture the structure..." The method is deliberate sabotage: "You take a piece of text, you corrupt it in some way, by for example removing some words, and then you train a big neural net to predict the words that are missing." And one narrow version of that trick runs every chatbot on Earth: "A special case of this is that you take a piece of text and the last word in that text is not visible, and so you train the system to predict the last word in that text — and this is the way large language models are trained on." So why did the third one win? Supervised learning needs a human. Reinforcement learning needs a crash. Self-supervised learning needs neither — because the missing word and the correct answer are the same thing. The data grades itself.

Former Meta Chief AI Scientist Yann LeCun on the three paradigms of machine learning — and why the third is what made ChatGPT possible: Here's each one, and where it breaks. First, supervised learning. You tell the machine the answer. "You show it a picture, let's say of a table, and you tell it this is a table. So it's supervised because you tell it what the correct answer is." Get it wrong, and the machine rewrites itself: "The system computes its output, and if it says something else than table, then it's going to adjust its parameters, its internal structure, so that the output it produces gets closer to the output you want." Repeat at scale and something more than memorisation appears: "Eventually the system will find a way to recognize every image you trained it on, but also images it's never seen that are similar to the one you train it on. This is called a generalization ability." The limit: a human has to supply every single answer. That doesn't scale to the size of the internet. Second, reinforcement learning. You don't give the answer, only a verdict. "You don't tell the system what the correct answer is. You only tell it whether the answer it produced was good or bad." Learning to ride a bike, essentially: "You try to ride a bike and you don't know how to ride the bike and after a while you fall. So you know you did something bad and so you change your strategy a little bit. And eventually you learn how to ride a bike." For years the field assumed this was the closest thing to how animals actually learn. Yann LeCun's verdict: "Now it turns out reinforcement learning is extremely inefficient." It dominates wherever failure is free: "It works really well if you want to train a system to play chess or play go or poker, because you can have the system play millions and millions of games against itself and basically fine-tune itself. But it doesn't really work in the real world." The limit, in one image: "If you want to train a car to drive itself, you're not going to do it with reinforcement learning. It's going to crash thousands of times." On robotics he's careful rather than dismissive: "Reinforcement learning can be part of the solution, but it's not the complete answer. It's not sufficient." Third, self-supervised learning. You tell the machine nothing at all. "And this is what has enabled the recent progress in natural language understanding and chatbots." The strange part is that you stop asking for a task: "You don't train the system to accomplish any particular task. You just train it to basically capture the structure..." The method is deliberate sabotage: "You take a piece of text, you corrupt it in some way, by for example removing some words, and then you train a big neural net to predict the words that are missing." And one narrow version of that trick runs every chatbot on Earth: "A special case of this is that you take a piece of text and the last word in that text is not visible, and so you train the system to predict the last word in that text — and this is the way large language models are trained on." So why did the third one win? Supervised learning needs a human. Reinforcement learning needs a crash. Self-supervised learning needs neither — because the missing word and the correct answer are the same thing. The data grades itself.

Big Brain AI

49,293 görüntüleme • 2 ay önce

IN 1986 MIT FILMED THE LECTURE WHERE CODE STOPPED BEING CODE AND BECAME DATA 43 minutes from Gerald Sussman, in the most legendary intro programming course ever recorded. -> The idea that lands: to a program, your code is just data it can read and rewrite. He builds a program that does real calculus -- not by crunching numbers, but by reading the math as a list and transforming it piece by piece. Then he shows the trick hiding under all of it: in this language, code and data are the same material. Forty years later that is exactly what an AI coding agent is -- a program that reads your code as data, rewrites it, and hands it back. Sussman drew the whole idea on a blackboard in 1986. Writing code was never the deepest skill -- understanding that code itself is data a program can manipulate is. This is where you learn it. Most people think AI writing code is brand new. The ones who watch this saw the blueprint 40 years ago. Bookmark & Watch it. This one's a legend ↓

slash1s

22,107 görüntüleme • 4 ay önce

New PNAS paper. Historical GDP per capita data is scarce, but data on the places of birth, death, and occupations of famous individuals is abundant. In this paper we estimate the historical GDP per capita of hundreds of regions in Europe and North America using a machine learning model that leveraged data on about 500k famous biographies. Our estimates more-or-less quadruple the availability of historical GDP per capita estimates for the last 700 years. So why use biographies to augment historical GDP per capita data? Biographical data contains information about people who might have contributed directly to economic growth, like James Watt, or that were attracted to wealthy places looking for patrons, like Michelangelo. So we--mainly Philipp (Philipp Koch)--used this data to construct hundreds of features describing each European region. Then, we trained a machine learning model to find the features that explained most of the variance in a cross-validation test, where we split regions multiple times into a training set and a test set. On average, the model explained about 90% of the variance in GDP per capita of the regions it had not seen during training. But we wanted to go further, and Philipp really went to town by looking at different ways to validate our estimates. We found our estimates correlate positively with historical measures of wellbeing, church building activity, urbanization, and body height. We also used these measures to reproduce the basic Atlantic trade result of Acemoglu, Johnson, and Robison and to explore the economic consequences of the famous Lisbon earthquake of 1755. But what I personally loved most about this project, other than working with Philipp Koch and V, is that it shows that we can use machine learning methods not only to explore the future, but the past. There is a bright and growing future in the use of machine learning for economic history. Hope you enjoy the paper and the data. You can find links to the paper and a data exploration tool in the first comment.

New PNAS paper. Historical GDP per capita data is scarce, but data on the places of birth, death, and occupations of famous individuals is abundant. In this paper we estimate the historical GDP per capita of hundreds of regions in Europe and North America using a machine learning model that leveraged data on about 500k famous biographies. Our estimates more-or-less quadruple the availability of historical GDP per capita estimates for the last 700 years. So why use biographies to augment historical GDP per capita data? Biographical data contains information about people who might have contributed directly to economic growth, like James Watt, or that were attracted to wealthy places looking for patrons, like Michelangelo. So we--mainly Philipp (Philipp Koch)--used this data to construct hundreds of features describing each European region. Then, we trained a machine learning model to find the features that explained most of the variance in a cross-validation test, where we split regions multiple times into a training set and a test set. On average, the model explained about 90% of the variance in GDP per capita of the regions it had not seen during training. But we wanted to go further, and Philipp really went to town by looking at different ways to validate our estimates. We found our estimates correlate positively with historical measures of wellbeing, church building activity, urbanization, and body height. We also used these measures to reproduce the basic Atlantic trade result of Acemoglu, Johnson, and Robison and to explore the economic consequences of the famous Lisbon earthquake of 1755. But what I personally loved most about this project, other than working with Philipp Koch and V, is that it shows that we can use machine learning methods not only to explore the future, but the past. There is a bright and growing future in the use of machine learning for economic history. Hope you enjoy the paper and the data. You can find links to the paper and a data exploration tool in the first comment.

César A. Hidalgo

54,365 görüntüleme • 2 yıl önce

Paper mesh deformation with pure 2D canvas API. The sheet is built as a grid of points, not as one flat image. Each point keeps its base position, current position, previous position, and a small fake depth value. When I drag one area, the code pulls the nearest point and spreads that force through the grid, so the paper bends instead of just skewing like a card. The structure is held together with constraints between points. Horizontal, vertical, diagonal, and longer links all try to stay close to their original length. That is what gives it paper-like resistance. It can flex, but it does not stretch too far or collapse too much. The solver runs many times per frame, which keeps the shape stable while dragging. The fold detail is driven by texture data. One image is the visible paper surface. Another image is sampled as a fold map: dark values become stronger fold areas, and image gradients give fold direction. That data feeds back into the mesh, so some parts lift more, crease more, and react with more stiffness than others. The rendering is still just 2D canvas. Each cell in the grid is split into triangles, and every triangle is drawn with its own transform so the texture stays locked to the surface while it deforms. Then a shading pass adds highlights and shadows from the fake depth and crease values. No WebGL, no shaders. Just mesh math, texture sampling.

Paper mesh deformation with pure 2D canvas API. The sheet is built as a grid of points, not as one flat image. Each point keeps its base position, current position, previous position, and a small fake depth value. When I drag one area, the code pulls the nearest point and spreads that force through the grid, so the paper bends instead of just skewing like a card. The structure is held together with constraints between points. Horizontal, vertical, diagonal, and longer links all try to stay close to their original length. That is what gives it paper-like resistance. It can flex, but it does not stretch too far or collapse too much. The solver runs many times per frame, which keeps the shape stable while dragging. The fold detail is driven by texture data. One image is the visible paper surface. Another image is sampled as a fold map: dark values become stronger fold areas, and image gradients give fold direction. That data feeds back into the mesh, so some parts lift more, crease more, and react with more stiffness than others. The rendering is still just 2D canvas. Each cell in the grid is split into triangles, and every triangle is drawn with its own transform so the texture stays locked to the surface while it deforms. Then a shading pass adds highlights and shadows from the fake depth and crease values. No WebGL, no shaders. Just mesh math, texture sampling.

Alex Barashkov

17,732 görüntüleme • 7 ay önce

In 1998, Japanese Mathematician, Engineer and Machine Learning pioneer Shun-ichi Amari published "Natural Gradient Works Efficiently in Learning (722 Cites in Papers and 9 Cites in Patents)" and made a point that still feels fresh even today If your loss is L(θ), then the usual update θ̇ = −∇ L(θ) quietly assumes your parameter space is flat. Amari asked what happens when the model itself has geometry, described by G(θ), the Fisher information. Then the more natural direction becomes θ̇ = −G(θ)⁻¹∇ L(θ). This is the same loss, but with a different notion of distance. As the animation shows, ordinary Gradient Descent follows the raw slope and gets dragged around by distortion, while the natural gradient moves in a way that respects the geometry of the statistical model itself.

In 1998, Japanese Mathematician, Engineer and Machine Learning pioneer Shun-ichi Amari published "Natural Gradient Works Efficiently in Learning (722 Cites in Papers and 9 Cites in Patents)" and made a point that still feels fresh even today If your loss is L(θ), then the usual update θ̇ = −∇ L(θ) quietly assumes your parameter space is flat. Amari asked what happens when the model itself has geometry, described by G(θ), the Fisher information. Then the more natural direction becomes θ̇ = −G(θ)⁻¹∇ L(θ). This is the same loss, but with a different notion of distance. As the animation shows, ordinary Gradient Descent follows the raw slope and gets dragged around by distortion, while the natural gradient moves in a way that respects the geometry of the statistical model itself.

Mathelirium

45,802 görüntüleme • 7 ay önce

Stanford Prof. Christopher Manning says today’s AI can absorb 2,000 years of human knowledge and still be far worse than humans at learning from just a few examples: "No matter how fantastic these systems are, there's still a lot we can't do. A lot of it looks better than it is because these huge models have been able to slurp up 2,000-plus years of human learning across all domains and stick it all in one huge neural net." "That's amazing because it knows everything to a first approximation. But really that's exploiting hard-won human knowledge, and that's different to the large language model discovering its own knowledge the same way human beings have been for 2,000 years." "At the end of the day, the secret of human intelligence is actually the speed at which we can learn. We only need to see a couple of examples of something, and we can learn something and intuit it." "Our current AI is nothing like that. We're substituting, here are 10,000 examples of this thing, because really the machine learning is very poor compared to human learning." Stanford NLP Group Stanford University

Stanford Prof. Christopher Manning says today’s AI can absorb 2,000 years of human knowledge and still be far worse than humans at learning from just a few examples: "No matter how fantastic these systems are, there's still a lot we can't do. A lot of it looks better than it is because these huge models have been able to slurp up 2,000-plus years of human learning across all domains and stick it all in one huge neural net." "That's amazing because it knows everything to a first approximation. But really that's exploiting hard-won human knowledge, and that's different to the large language model discovering its own knowledge the same way human beings have been for 2,000 years." "At the end of the day, the secret of human intelligence is actually the speed at which we can learn. We only need to see a couple of examples of something, and we can learn something and intuit it." "Our current AI is nothing like that. We're substituting, here are 10,000 examples of this thing, because really the machine learning is very poor compared to human learning." Stanford NLP Group Stanford University

MTS

16,405 görüntüleme • 1 ay önce

Geometry of Machine Learning Models - Gaussian Process Kernel In 1948 Norbert Wiener framed prediction as a correlation problem, and in the 1970s George Wahba clarified that picking a smoothness preference is the same as picking a kernel. The motivation is that whenever data i s sparse and noisy, you don't just want a fit, you want a fit with calibrated uncertainty.

Geometry of Machine Learning Models - Gaussian Process Kernel In 1948 Norbert Wiener framed prediction as a correlation problem, and in the 1970s George Wahba clarified that picking a smoothness preference is the same as picking a kernel. The motivation is that whenever data i s sparse and noisy, you don't just want a fit, you want a fit with calibrated uncertainty.

Mathelirium

15,661 görüntüleme • 7 ay önce

Do you know how we simulate electromagnetic radiation around real systems like antennas, aircraft, and waveguides? We break the geometry into a mesh, a collection of small elements that lets the solver compute Maxwell’s equations locally instead of attacking the whole shape at once. This makes the simulations practical, fast, and accurate enough to use in real engineering.

Do you know how we simulate electromagnetic radiation around real systems like antennas, aircraft, and waveguides? We break the geometry into a mesh, a collection of small elements that lets the solver compute Maxwell’s equations locally instead of attacking the whole shape at once. This makes the simulations practical, fast, and accurate enough to use in real engineering.

Mathelirium

73,166 görüntüleme • 6 ay önce

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Alex Ker 🔭

1,367,231 görüntüleme • 3 yıl önce

I got a smart meter recently and saw you can download a HDF file with the data so I had the idea of writing a script that could parse that and show the data in a useful manner. However I discovered that someone has already done this and done a really good job on it. The video explains how it works. In simple terms it uses the ESB smart meter data and shows a breakdown of how much data you are using and when and also recommends plans and estimates what each one would cost based on the data. The tool is available at easier to read on Laptop or Tablet or then your phone to Landscape.

I got a smart meter recently and saw you can download a HDF file with the data so I had the idea of writing a script that could parse that and show the data in a useful manner. However I discovered that someone has already done this and done a really good job on it. The video explains how it works. In simple terms it uses the ESB smart meter data and shows a breakdown of how much data you are using and when and also recommends plans and estimates what each one would cost based on the data. The tool is available at easier to read on Laptop or Tablet or then your phone to Landscape.

Carlow Weather

298,565 görüntüleme • 2 yıl önce

AI runs on electricity, and Sweden has a head start. This morning at Ruinen we put the question on the table: is the next wave of AI data centres a Swedish opportunity, and what does the grid need to make it happen?

AI runs on electricity, and Sweden has a head start. This morning at Ruinen we put the question on the table: is the next wave of AI data centres a Swedish opportunity, and what does the grid need to make it happen?

Hitachi Energy

6,945,271 görüntüleme • 4 ay önce

A Neural Network Can Grow New Neurons Where It Is Confused? In 1994, Bernd Fritzke published A Growing Neural Gas Network Learns Topologies. He introduced a network that starts small, follows incoming data, and inserts new neurons where its error is highest. In the animation, the fog is the drifting data. The glowing nodes are neurons. The fibers are learned connections. The network grows into a living skeleton of the manifold.

A Neural Network Can Grow New Neurons Where It Is Confused? In 1994, Bernd Fritzke published A Growing Neural Gas Network Learns Topologies. He introduced a network that starts small, follows incoming data, and inserts new neurons where its error is highest. In the animation, the fog is the drifting data. The glowing nodes are neurons. The fibers are learned connections. The network grows into a living skeleton of the manifold.

Mathelirium

38,431 görüntüleme • 5 ay önce

A Self-Organizing Map Learns The Shape of Data

A Self-Organizing Map Learns The Shape of Data

Mathelirium

44,616 görüntüleme • 8 ay önce