正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision.

ₕₐₘₚₜₒₙ

217,741 subscribers

218,755 次观看 • 2 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 条评论

Roy van Rijn 的头像

Roy van Rijn2 年前

Or…. it could be a single spline 🙄

ΣTΞCH 的头像

ΣTΞCH2 年前

If you want 25 minutes of how to make a universal function approximation, there's a great video by this one guy who gave it a shot with a handful of methods. Really cool to observe the thinking behind it

BrokieRichard 的头像

BrokieRichard2 年前

You spelled Stone-Weierstrass wrong

ऋषिक तिवारी (Rishik Tiwari) 的头像

ऋषिक तिवारी (Rishik Tiwari)2 年前

You are talking about general curve fitting (all NNs do) but are wrong about any desired precision. The precision is directly proportional to width of the single layer. Which also means if you are dealing with periodic functions, then nyquist sampling theorem shall apply.

mel 🐰 的头像

mel 🐰2 年前

Omg this is the thing they make me draw in math class…

HL®H 的头像

HL®H2 年前

“The Universal Approximation Theorem means that a simple neural network can accurately model any continuous function within a certain set, as long as it has enough neurons. It's like a versatile tool for many tasks.”

A$AP 的头像

A$AP2 年前

Is this a consequence that continuous functions can be approximated by piecewise functions ?

Jesse Palmer 的头像

Jesse Palmer2 年前

Does that mean multiple hidden layers handle discontinuity?

Matthew Zeits 的头像

Matthew Zeits2 年前

How about functions mapping vectors to vectors? Or complex vectors?

; 的头像

;2 年前

How does the architecture of a neural network change when incorporating multiple hidden layers, and how does this relate to the universal approximation theorem?

相关视频

Oldies but goldies: A. Barron, Universal Approximation Bounds for Superpositions of a Sigmoidal Function, 1993. Proves that 1 hidden layer perceptrons break the curse of dimensionality to approximate a class of smooth functions.

Oldies but goldies: A. Barron, Universal Approximation Bounds for Superpositions of a Sigmoidal Function, 1993. Proves that 1 hidden layer perceptrons break the curse of dimensionality to approximate a class of smooth functions.

Gabriel Peyré

52,682 次观看 • 2 年前

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

Jesús Fernández-Villaverde

46,908 次观看 • 2 个月前

The next interface isn’t a screen — it’s neural. PL network team Precision Neuroscience is working at the intersection of neuroscience and digital interaction, exploring how neural signals can help people with neurodegenerative diseases.

The next interface isn’t a screen — it’s neural. PL network team Precision Neuroscience is working at the intersection of neuroscience and digital interaction, exploring how neural signals can help people with neurodegenerative diseases.

Protocol Labs

338,864 次观看 • 5 个月前

Demis Hassabis says our brains are likely to be approximate Turing machines. AlphaFold showed why that matters: protein folding looked like a quantum problem, but a classical neural network could model it well enough to solve it. The world may be quantum, but understanding it may only need the right approximation.

Demis Hassabis says our brains are likely to be approximate Turing machines. AlphaFold showed why that matters: protein folding looked like a quantum problem, but a classical neural network could model it well enough to solve it. The world may be quantum, but understanding it may only need the right approximation.

vitrupo

47,068 次观看 • 2 个月前

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

31,267 次观看 • 9 个月前

A costume designer’s (Lebannen) opinion of Flins is that his outfit is based on a formal attire: A layered look: a V-neck outer layer with a metallic “tie”. The outer layer has a trench coat with fitted pants underneath and high boots. The trench coat functions as a suit, and

A costume designer’s (Lebannen) opinion of Flins is that his outfit is based on a formal attire: A layered look: a V-neck outer layer with a metallic “tie”. The outer layer has a trench coat with fitted pants underneath and high boots. The trench coat functions as a suit, and

Timely Flins Lore

29,307 次观看 • 2 个月前

Meet Raymond Dong from Avantis From a love of building that started with Legos, to building Avantis - the universal leverage layer on Base This is his story

Meet Raymond Dong from Avantis From a love of building that started with Legos, to building Avantis - the universal leverage layer on Base This is his story

Base

926,924 次观看 • 2 个月前

Fundamental Theorem of Calculus 2 has a lovely visual. Imagine you're climbing a mountain. Add up all the little changes in height as you walk up. You can approximate those little changes with the derivative. Then the sum of the little changes adds up to the net change.

Fundamental Theorem of Calculus 2 has a lovely visual. Imagine you're climbing a mountain. Add up all the little changes in height as you walk up. You can approximate those little changes with the derivative. Then the sum of the little changes adds up to the net change.

Trefor Bazett

23,517 次观看 • 2 个月前

Geoffrey Hinton says you can't prove what a neural network will do, just like you can't prove a taxi driver won't kill you Once trained, they become huge sets of weights that cannot be fully explained or guaranteed "the best we can do with AI is run good safety tests and trust the data"

Geoffrey Hinton says you can't prove what a neural network will do, just like you can't prove a taxi driver won't kill you Once trained, they become huge sets of weights that cannot be fully explained or guaranteed "the best we can do with AI is run good safety tests and trust the data"

Haider.

14,895 次观看 • 4 个月前

Elon Musk: You can solve any amount of traffic with a 3D network of tunnels. “There's no real limit to how many levels of tunnel you can have. You can go much further deep than you can go up. The deepest mines are much deeper than the tallest buildings are tall, so you can alleviate any arbitrary level of urban congestion with a 3D tunnel network. This is a very important point. A key rebuttal to the tunnels is that if you add one layer of tunnels, that will simply alleviate congestion, it will get used up, and then you'll be back where you started, back with congestion. But you can go to any arbitrary number of tunnels, any number of levels.” Source: TED, April 28, 2017.

Elon Musk: You can solve any amount of traffic with a 3D network of tunnels. “There's no real limit to how many levels of tunnel you can have. You can go much further deep than you can go up. The deepest mines are much deeper than the tallest buildings are tall, so you can alleviate any arbitrary level of urban congestion with a 3D tunnel network. This is a very important point. A key rebuttal to the tunnels is that if you add one layer of tunnels, that will simply alleviate congestion, it will get used up, and then you'll be back where you started, back with congestion. But you can go to any arbitrary number of tunnels, any number of levels.” Source: TED, April 28, 2017.

ELON CLIPS

18,258 次观看 • 1 个月前

This is the Meltio Engine, a 3D printer that creates metal parts with high precision. It uses laser-based technology with wire and powder together to build complex parts layer by layer. It is faster than traditional methods and can also repair or upgrade existing metal parts.

This is the Meltio Engine, a 3D printer that creates metal parts with high precision. It uses laser-based technology with wire and powder together to build complex parts layer by layer. It is faster than traditional methods and can also repair or upgrade existing metal parts.

Space and Technology

48,697 次观看 • 3 个月前

TrustWare was selected as a Codebase Season 4 honoree. They’re building a universal deposit layer that lets apps accept any asset on any chain in one click, while they handle the bridges, swaps, and routing. trustware will receive a grant + ongoing support from Ava Labs.

TrustWare was selected as a Codebase Season 4 honoree. They’re building a universal deposit layer that lets apps accept any asset on any chain in one click, while they handle the bridges, swaps, and routing. trustware will receive a grant + ongoing support from Ava Labs.

Avalanche🔺

15,771 次观看 • 5 个月前

We just connected every blockchain on CCIP. Introducing XLine - one infrastructure layer connecting the entire CCIP network. Where a route between two chains exists, XLine finds the best one. Where it doesn't, XLine builds it. Any token, any chain. One solution.

We just connected every blockchain on CCIP. Introducing XLine - one infrastructure layer connecting the entire CCIP network. Where a route between two chains exists, XLine finds the best one. Where it doesn't, XLine builds it. Any token, any chain. One solution.

XSwap 🔗

14,659 次观看 • 1 个月前

can a neural network learn to walk as a physical object in a physics simulation? here I train walking neural nets with an evolutionary algorithm. The input nodes/feet are activated by sine waves at learned phases & connections between two neurons extend based on their difference

can a neural network learn to walk as a physical object in a physics simulation? here I train walking neural nets with an evolutionary algorithm. The input nodes/feet are activated by sine waves at learned phases & connections between two neurons extend based on their difference

Matt Henderson

339,340 次观看 • 3 年前

MSNBC, which just had one of its producers caught on hidden camera admitting the network is indistinguishable from the Democrat Party, says with a straight face that Elon Musk is the “world’s leading spreader of disinformation.” HAHAHAHAHAHAHAHA

MSNBC, which just had one of its producers caught on hidden camera admitting the network is indistinguishable from the Democrat Party, says with a straight face that Elon Musk is the “world’s leading spreader of disinformation.” HAHAHAHAHAHAHAHA

Charlie Kirk

1,453,481 次观看 • 1 年前

In the era of 10T-parameter foundation models 😈, can we still build next-generation world representations using 🦥 a single RTX 3090 GPU? We introduce WorldString (a name inspired by string theory), a tiny neural network representation for reconstructing dynamic world instances, ranging from articulated motion and skeletal skinning to soft-body deformation. 🔥 We view object representation as the core primitive of world modeling: the basic executable unit from which perception, interaction, simulation, and generation can be built. With a compact neural basis, WorldString aims to connect dynamic object modeling with neural simulation and foundation models, enabling a scalable path toward richer representations of the physical world.

In the era of 10T-parameter foundation models 😈, can we still build next-generation world representations using 🦥 a single RTX 3090 GPU? We introduce WorldString (a name inspired by string theory), a tiny neural network representation for reconstructing dynamic world instances, ranging from articulated motion and skeletal skinning to soft-body deformation. 🔥 We view object representation as the core primitive of world modeling: the basic executable unit from which perception, interaction, simulation, and generation can be built. With a compact neural basis, WorldString aims to connect dynamic object modeling with neural simulation and foundation models, enabling a scalable path toward richer representations of the physical world.

Xueyan Zou

38,299 次观看 • 1 个月前

DynamicVLA A compact 0.4B Vision-Language-Action model that finally lets robots manipulate *moving* objects in real-time, closing the perception-execution gap with Continuous Inference and Latent-aware Action Streaming.

DynamicVLA A compact 0.4B Vision-Language-Action model that finally lets robots manipulate moving objects in real-time, closing the perception-execution gap with Continuous Inference and Latent-aware Action Streaming.

DailyPapers

16,357 次观看 • 6 个月前

Privacy on the Stellar network balances the needs of institutional finance with the mechanics that make blockchain valuable. Start with an open base layer. Add privacy controls on top. Denelle Dixon on Tony Edward (Thinking Crypto Podcast).

Privacy on the Stellar network balances the needs of institutional finance with the mechanics that make blockchain valuable. Start with an open base layer. Add privacy controls on top. Denelle Dixon on Tony Edward (Thinking Crypto Podcast).

Stellar

38,661 次观看 • 18 天前

Builders design space shouldn’t be boxed in by network limitations or fragmented tech stacks. Shido Network is the ultimate developer sandbox, supporting multiple VMs with zero downtime and infinite scalability. Build the impossible on a Layer-1 foundation that actually keeps up with your vision.

Builders design space shouldn’t be boxed in by network limitations or fragmented tech stacks. Shido Network is the ultimate developer sandbox, supporting multiple VMs with zero downtime and infinite scalability. Build the impossible on a Layer-1 foundation that actually keeps up with your vision.

Shido

139,649 次观看 • 4 个月前