Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision.

ₕₐₘₚₜₒₙ

216,644 subscribers

218,753 görüntüleme • 2 yıl önce •via X (Twitter)

Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

10 Yorum

Roy van Rijn profil fotoğrafı

Roy van Rijn2 yıl önce

Or…. it could be a single spline 🙄

ΣTΞCH profil fotoğrafı

ΣTΞCH2 yıl önce

If you want 25 minutes of how to make a universal function approximation, there's a great video by this one guy who gave it a shot with a handful of methods. Really cool to observe the thinking behind it

BrokieRichard profil fotoğrafı

BrokieRichard2 yıl önce

You spelled Stone-Weierstrass wrong

ऋषिक तिवारी (Rishik Tiwari) profil fotoğrafı

ऋषिक तिवारी (Rishik Tiwari)2 yıl önce

You are talking about general curve fitting (all NNs do) but are wrong about any desired precision. The precision is directly proportional to width of the single layer. Which also means if you are dealing with periodic functions, then nyquist sampling theorem shall apply.

mel 🐰 profil fotoğrafı

mel 🐰2 yıl önce

Omg this is the thing they make me draw in math class…

HL®H profil fotoğrafı

HL®H2 yıl önce

“The Universal Approximation Theorem means that a simple neural network can accurately model any continuous function within a certain set, as long as it has enough neurons. It's like a versatile tool for many tasks.”

A$AP profil fotoğrafı

A$AP2 yıl önce

Is this a consequence that continuous functions can be approximated by piecewise functions ?

Jesse Palmer profil fotoğrafı

Jesse Palmer2 yıl önce

Does that mean multiple hidden layers handle discontinuity?

Matthew Zeits profil fotoğrafı

Matthew Zeits2 yıl önce

How about functions mapping vectors to vectors? Or complex vectors?

; profil fotoğrafı

;2 yıl önce

How does the architecture of a neural network change when incorporating multiple hidden layers, and how does this relate to the universal approximation theorem?

Benzer Videolar

Oldies but goldies: A. Barron, Universal Approximation Bounds for Superpositions of a Sigmoidal Function, 1993. Proves that 1 hidden layer perceptrons break the curse of dimensionality to approximate a class of smooth functions.

Oldies but goldies: A. Barron, Universal Approximation Bounds for Superpositions of a Sigmoidal Function, 1993. Proves that 1 hidden layer perceptrons break the curse of dimensionality to approximate a class of smooth functions.

Gabriel Peyré

52,682 görüntüleme • 2 yıl önce

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

Jesús Fernández-Villaverde

46,690 görüntüleme • 1 ay önce

The next interface isn’t a screen — it’s neural. PL network team Precision Neuroscience is working at the intersection of neuroscience and digital interaction, exploring how neural signals can help people with neurodegenerative diseases.

The next interface isn’t a screen — it’s neural. PL network team Precision Neuroscience is working at the intersection of neuroscience and digital interaction, exploring how neural signals can help people with neurodegenerative diseases.

Protocol Labs

338,864 görüntüleme • 4 ay önce

Demis Hassabis says our brains are likely to be approximate Turing machines. AlphaFold showed why that matters: protein folding looked like a quantum problem, but a classical neural network could model it well enough to solve it. The world may be quantum, but understanding it may only need the right approximation.

Demis Hassabis says our brains are likely to be approximate Turing machines. AlphaFold showed why that matters: protein folding looked like a quantum problem, but a classical neural network could model it well enough to solve it. The world may be quantum, but understanding it may only need the right approximation.

vitrupo

47,068 görüntüleme • 1 ay önce

Can you calculate a one-node neural network by hand ✍️

Can you calculate a one-node neural network by hand ✍️

Tom Yeh

10,362 görüntüleme • 6 ay önce

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

26,930 görüntüleme • 7 ay önce

A costume designer’s (Lebannen) opinion of Flins is that his outfit is based on a formal attire: A layered look: a V-neck outer layer with a metallic “tie”. The outer layer has a trench coat with fitted pants underneath and high boots. The trench coat functions as a suit, and

A costume designer’s (Lebannen) opinion of Flins is that his outfit is based on a formal attire: A layered look: a V-neck outer layer with a metallic “tie”. The outer layer has a trench coat with fitted pants underneath and high boots. The trench coat functions as a suit, and

Timely Flins Lore

29,307 görüntüleme • 1 ay önce

Meet Raymond Dong from Avantis From a love of building that started with Legos, to building Avantis - the universal leverage layer on Base This is his story

Meet Raymond Dong from Avantis From a love of building that started with Legos, to building Avantis - the universal leverage layer on Base This is his story

Base

920,715 görüntüleme • 28 gün önce

Geoffrey Hinton says you can't prove what a neural network will do, just like you can't prove a taxi driver won't kill you Once trained, they become huge sets of weights that cannot be fully explained or guaranteed "the best we can do with AI is run good safety tests and trust the data"

Geoffrey Hinton says you can't prove what a neural network will do, just like you can't prove a taxi driver won't kill you Once trained, they become huge sets of weights that cannot be fully explained or guaranteed "the best we can do with AI is run good safety tests and trust the data"

Haider.

14,895 görüntüleme • 3 ay önce

Fundamental Theorem of Calculus 2 has a lovely visual. Imagine you're climbing a mountain. Add up all the little changes in height as you walk up. You can approximate those little changes with the derivative. Then the sum of the little changes adds up to the net change.

Fundamental Theorem of Calculus 2 has a lovely visual. Imagine you're climbing a mountain. Add up all the little changes in height as you walk up. You can approximate those little changes with the derivative. Then the sum of the little changes adds up to the net change.

Trefor Bazett

23,387 görüntüleme • 1 ay önce

Hopfield recall Spiking Neural network now this one is cool isn't it? -open source, link in comment- This is a Spiking Neural Network (SNN) that can learn and recall patterns through Hebbian learning, similar to a Hopfield network but using biologically-inspired spiking dynamics rather than energy-based settling.

Hopfield recall Spiking Neural network now this one is cool isn't it? -open source, link in comment- This is a Spiking Neural Network (SNN) that can learn and recall patterns through Hebbian learning, similar to a Hopfield network but using biologically-inspired spiking dynamics rather than energy-based settling.

echo.hive

31,452 görüntüleme • 5 ay önce

This is the Meltio Engine, a 3D printer that creates metal parts with high precision. It uses laser-based technology with wire and powder together to build complex parts layer by layer. It is faster than traditional methods and can also repair or upgrade existing metal parts.

This is the Meltio Engine, a 3D printer that creates metal parts with high precision. It uses laser-based technology with wire and powder together to build complex parts layer by layer. It is faster than traditional methods and can also repair or upgrade existing metal parts.

Space and Technology

48,653 görüntüleme • 2 ay önce

Stream: The universal layer powering the future of content. Are you with us?

Stream: The universal layer powering the future of content. Are you with us?

Stream

387,182 görüntüleme • 2 yıl önce

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 görüntüleme • 1 yıl önce

With OpenAI, Figure 01 can now have full conversations with people -OpenAI models provide high-level visual and language intelligence -Figure neural networks deliver fast, low-level, dexterous robot actions Everything in this video is a neural network:

With OpenAI, Figure 01 can now have full conversations with people -OpenAI models provide high-level visual and language intelligence -Figure neural networks deliver fast, low-level, dexterous robot actions Everything in this video is a neural network:

Figure

5,100,314 görüntüleme • 2 yıl önce

can a neural network learn to walk as a physical object in a physics simulation? here I train walking neural nets with an evolutionary algorithm. The input nodes/feet are activated by sine waves at learned phases & connections between two neurons extend based on their difference

can a neural network learn to walk as a physical object in a physics simulation? here I train walking neural nets with an evolutionary algorithm. The input nodes/feet are activated by sine waves at learned phases & connections between two neurons extend based on their difference

Matt Henderson

339,145 görüntüleme • 3 yıl önce

MSNBC, which just had one of its producers caught on hidden camera admitting the network is indistinguishable from the Democrat Party, says with a straight face that Elon Musk is the “world’s leading spreader of disinformation.” HAHAHAHAHAHAHAHA

MSNBC, which just had one of its producers caught on hidden camera admitting the network is indistinguishable from the Democrat Party, says with a straight face that Elon Musk is the “world’s leading spreader of disinformation.” HAHAHAHAHAHAHAHA

Charlie Kirk

1,453,481 görüntüleme • 1 yıl önce

DynamicVLA A compact 0.4B Vision-Language-Action model that finally lets robots manipulate *moving* objects in real-time, closing the perception-execution gap with Continuous Inference and Latent-aware Action Streaming.

DynamicVLA A compact 0.4B Vision-Language-Action model that finally lets robots manipulate moving objects in real-time, closing the perception-execution gap with Continuous Inference and Latent-aware Action Streaming.

DailyPapers

16,291 görüntüleme • 4 ay önce

In the era of 10T-parameter foundation models 😈, can we still build next-generation world representations using 🦥 a single RTX 3090 GPU? We introduce WorldString (a name inspired by string theory), a tiny neural network representation for reconstructing dynamic world instances, ranging from articulated motion and skeletal skinning to soft-body deformation. 🔥 We view object representation as the core primitive of world modeling: the basic executable unit from which perception, interaction, simulation, and generation can be built. With a compact neural basis, WorldString aims to connect dynamic object modeling with neural simulation and foundation models, enabling a scalable path toward richer representations of the physical world.

In the era of 10T-parameter foundation models 😈, can we still build next-generation world representations using 🦥 a single RTX 3090 GPU? We introduce WorldString (a name inspired by string theory), a tiny neural network representation for reconstructing dynamic world instances, ranging from articulated motion and skeletal skinning to soft-body deformation. 🔥 We view object representation as the core primitive of world modeling: the basic executable unit from which perception, interaction, simulation, and generation can be built. With a compact neural basis, WorldString aims to connect dynamic object modeling with neural simulation and foundation models, enabling a scalable path toward richer representations of the physical world.

Xueyan Zou

36,081 görüntüleme • 4 gün önce