Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision.

hamptonism's profile picture

ₕₐₘₚₜₒₙ

217,741 subscribers

218,755 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Roy van Rijn
Roy van Rijnvor 2 Jahren

Or…. it could be a single spline 🙄

Profilbild von ΣTΞCH
ΣTΞCHvor 2 Jahren

If you want 25 minutes of how to make a universal function approximation, there's a great video by this one guy who gave it a shot with a handful of methods. Really cool to observe the thinking behind it

Profilbild von BrokieRichard
BrokieRichardvor 2 Jahren

You spelled Stone-Weierstrass wrong

Profilbild von ऋषिक तिवारी (Rishik Tiwari)
ऋषिक तिवारी (Rishik Tiwari)vor 2 Jahren

You are talking about general curve fitting (all NNs do) but are wrong about any desired precision. The precision is directly proportional to width of the single layer. Which also means if you are dealing with periodic functions, then nyquist sampling theorem shall apply.

Profilbild von mel 🐰
mel 🐰vor 2 Jahren

Omg this is the thing they make me draw in math class…

Profilbild von HL®H
HL®Hvor 2 Jahren

“The Universal Approximation Theorem means that a simple neural network can accurately model any continuous function within a certain set, as long as it has enough neurons. It's like a versatile tool for many tasks.”

Profilbild von A$AP
A$APvor 2 Jahren

Is this a consequence that continuous functions can be approximated by piecewise functions ?

Profilbild von Jesse Palmer
Jesse Palmervor 2 Jahren

Does that mean multiple hidden layers handle discontinuity?

Profilbild von Matthew Zeits
Matthew Zeitsvor 2 Jahren

How about functions mapping vectors to vectors? Or complex vectors?

Profilbild von ;
;vor 2 Jahren

How does the architecture of a neural network change when incorporating multiple hidden layers, and how does this relate to the universal approximation theorem?

Ähnliche Videos

Oldies but goldies: A. Barron, Universal Approximation Bounds for Superpositions of a Sigmoidal Function, 1993. Proves that 1 hidden layer perceptrons break the curse of dimensionality to approximate a class of smooth functions.
0:20

Oldies but goldies: A. Barron, Universal Approximation Bounds for Superpositions of a Sigmoidal Function, 1993. Proves that 1 hidden layer perceptrons break the curse of dimensionality to approximate a class of smooth functions.

Gabriel Peyré

52,682 Aufrufe • vor 2 Jahren

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀
0:40

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

Jesús Fernández-Villaverde

46,908 Aufrufe • vor 2 Monaten

The next interface isn’t a screen — it’s neural. PL network team Precision Neuroscience is working at the intersection of neuroscience and digital interaction, exploring how neural signals can help people with neurodegenerative diseases.
0:25

The next interface isn’t a screen — it’s neural. PL network team Precision Neuroscience is working at the intersection of neuroscience and digital interaction, exploring how neural signals can help people with neurodegenerative diseases.

Protocol Labs

338,864 Aufrufe • vor 5 Monaten

Demis Hassabis says our brains are likely to be approximate Turing machines. AlphaFold showed why that matters: protein folding looked like a quantum problem, but a classical neural network could model it well enough to solve it. The world may be quantum, but understanding it may only need the right approximation.
0:55

Demis Hassabis says our brains are likely to be approximate Turing machines. AlphaFold showed why that matters: protein folding looked like a quantum problem, but a classical neural network could model it well enough to solve it. The world may be quantum, but understanding it may only need the right approximation.

vitrupo

47,068 Aufrufe • vor 2 Monaten

Can you calculate a one-node neural network by hand ✍️
2:46

Can you calculate a one-node neural network by hand ✍️

Tom Yeh

10,362 Aufrufe • vor 7 Monaten

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.
2:37

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

29,098 Aufrufe • vor 8 Monaten

AI developed at UMA Single neural network, End to End It operates robustly for hours, with precision, adapting to what it sees.
1:25

AI developed at UMA Single neural network, End to End It operates robustly for hours, with precision, adapting to what it sees.

Remi Cadene

51,376 Aufrufe • vor 5 Tagen

A costume designer’s (Lebannen) opinion of Flins is that his outfit is based on a formal attire: A layered look: a V-neck outer layer with a metallic “tie”. The outer layer has a trench coat with fitted pants underneath and high boots. The trench coat functions as a suit, and
2:07

A costume designer’s (Lebannen) opinion of Flins is that his outfit is based on a formal attire: A layered look: a V-neck outer layer with a metallic “tie”. The outer layer has a trench coat with fitted pants underneath and high boots. The trench coat functions as a suit, and

Timely Flins Lore

29,307 Aufrufe • vor 1 Monat

Meet Raymond Dong from Avantis From a love of building that started with Legos, to building Avantis - the universal leverage layer on Base This is his story
3:27

Meet Raymond Dong from Avantis From a love of building that started with Legos, to building Avantis - the universal leverage layer on Base This is his story

Base

924,940 Aufrufe • vor 1 Monat

Every layer of defence matters. EDGE’s VSHORAD solutions provide close-range aerial protection engineered for precision, speed, and reliability, because the final layer of defence can be the one that matters most.
0:38

Every layer of defence matters. EDGE’s VSHORAD solutions provide close-range aerial protection engineered for precision, speed, and reliability, because the final layer of defence can be the one that matters most.

EDGE

71,702 Aufrufe • vor 27 Tagen

Fundamental Theorem of Calculus 2 has a lovely visual. Imagine you're climbing a mountain. Add up all the little changes in height as you walk up. You can approximate those little changes with the derivative. Then the sum of the little changes adds up to the net change.
2:23

Fundamental Theorem of Calculus 2 has a lovely visual. Imagine you're climbing a mountain. Add up all the little changes in height as you walk up. You can approximate those little changes with the derivative. Then the sum of the little changes adds up to the net change.

Trefor Bazett

23,517 Aufrufe • vor 2 Monaten

Hopfield recall Spiking Neural network now this one is cool isn't it? -open source, link in comment- This is a Spiking Neural Network (SNN) that can learn and recall patterns through Hebbian learning, similar to a Hopfield network but using biologically-inspired spiking dynamics rather than energy-based settling.
1:07

Hopfield recall Spiking Neural network now this one is cool isn't it? -open source, link in comment- This is a Spiking Neural Network (SNN) that can learn and recall patterns through Hebbian learning, similar to a Hopfield network but using biologically-inspired spiking dynamics rather than energy-based settling.

echo.hive

31,452 Aufrufe • vor 5 Monaten

Geoffrey Hinton says you can't prove what a neural network will do, just like you can't prove a taxi driver won't kill you Once trained, they become huge sets of weights that cannot be fully explained or guaranteed "the best we can do with AI is run good safety tests and trust the data"
1:09

Geoffrey Hinton says you can't prove what a neural network will do, just like you can't prove a taxi driver won't kill you Once trained, they become huge sets of weights that cannot be fully explained or guaranteed "the best we can do with AI is run good safety tests and trust the data"

Haider.

14,895 Aufrufe • vor 3 Monaten

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N
1:15

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 Aufrufe • vor 1 Jahr

This is the Meltio Engine, a 3D printer that creates metal parts with high precision. It uses laser-based technology with wire and powder together to build complex parts layer by layer. It is faster than traditional methods and can also repair or upgrade existing metal parts.
0:43

This is the Meltio Engine, a 3D printer that creates metal parts with high precision. It uses laser-based technology with wire and powder together to build complex parts layer by layer. It is faster than traditional methods and can also repair or upgrade existing metal parts.

Space and Technology

48,697 Aufrufe • vor 3 Monaten

Elon Musk: You can solve any amount of traffic with a 3D network of tunnels. “There's no real limit to how many levels of tunnel you can have. You can go much further deep than you can go up. The deepest mines are much deeper than the tallest buildings are tall, so you can alleviate any arbitrary level of urban congestion with a 3D tunnel network. This is a very important point. A key rebuttal to the tunnels is that if you add one layer of tunnels, that will simply alleviate congestion, it will get used up, and then you'll be back where you started, back with congestion. But you can go to any arbitrary number of tunnels, any number of levels.” Source: TED, April 28, 2017.
0:36

Elon Musk: You can solve any amount of traffic with a 3D network of tunnels. “There's no real limit to how many levels of tunnel you can have. You can go much further deep than you can go up. The deepest mines are much deeper than the tallest buildings are tall, so you can alleviate any arbitrary level of urban congestion with a 3D tunnel network. This is a very important point. A key rebuttal to the tunnels is that if you add one layer of tunnels, that will simply alleviate congestion, it will get used up, and then you'll be back where you started, back with congestion. But you can go to any arbitrary number of tunnels, any number of levels.” Source: TED, April 28, 2017.

ELON CLIPS

18,192 Aufrufe • vor 14 Tagen

Stream: The universal layer powering the future of content. Are you with us?
0:20

Stream: The universal layer powering the future of content. Are you with us?

Stream

387,372 Aufrufe • vor 2 Jahren

TrustWare was selected as a Codebase Season 4 honoree. They’re building a universal deposit layer that lets apps accept any asset on any chain in one click, while they handle the bridges, swaps, and routing. trustware will receive a grant + ongoing support from Ava Labs.
2:10

TrustWare was selected as a Codebase Season 4 honoree. They’re building a universal deposit layer that lets apps accept any asset on any chain in one click, while they handle the bridges, swaps, and routing. trustware will receive a grant + ongoing support from Ava Labs.

Avalanche🔺

15,771 Aufrufe • vor 4 Monaten

With OpenAI, Figure 01 can now have full conversations with people -OpenAI models provide high-level visual and language intelligence -Figure neural networks deliver fast, low-level, dexterous robot actions Everything in this video is a neural network:
2:34

With OpenAI, Figure 01 can now have full conversations with people -OpenAI models provide high-level visual and language intelligence -Figure neural networks deliver fast, low-level, dexterous robot actions Everything in this video is a neural network:

Figure

5,101,403 Aufrufe • vor 2 Jahren

We just connected every blockchain on CCIP. Introducing XLine - one infrastructure layer connecting the entire CCIP network. Where a route between two chains exists, XLine finds the best one. Where it doesn't, XLine builds it. Any token, any chain. One solution.
0:32

We just connected every blockchain on CCIP. Introducing XLine - one infrastructure layer connecting the entire CCIP network. Where a route between two chains exists, XLine finds the best one. Where it doesn't, XLine builds it. Any token, any chain. One solution.

XSwap 🔗

14,346 Aufrufe • vor 12 Tagen