正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Many scientific problems hinge on finding interpretable formulas that fit data, but neural networks are the outright opposite! Check out our recent work that make neural networks modular and interpretable. If you have interesting datasets at hand, we're happy to collaborate!

Ziming Liu

14,295 subscribers

62,120 次观看 • 3 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 条评论

Ziming Liu 的头像

Ziming Liu3 年前

paper: code:

Ziming Liu 的头像

Ziming Liu3 年前

Symbolic regression (finding symbolic formulas from data) has been quite successful on some examples, but it also fail completely for others. Instead, our goal is structure regression (finding modular structures from data), which can provide visible insights for all cases.

Lei Chen 的头像

Lei Chen3 年前

Interesting work! Is this doable for graph nets? The task would be to locate significant attributed substructures. (I guess such location might be useful to help predesign the library of GSN @mmbronstein )

ExiledInfoHaz 的头像

ExiledInfoHaz3 年前

👀

Mitchell B. Slapik 的头像

Mitchell B. Slapik3 年前

Great work Ziming! Could you explain Figure 14? I was a bit confused by it.

Sam McKenzie 的头像

Sam McKenzie3 年前

We suffer in the seizure forecasting field making interpretable models because the features extracted from neural time series are highly correlated. Any tool to understand why our models work would give important insights as to how the brain changes in the lead up to a seizure.

bobert 的头像

bobert3 年前

I see some resemblance with Group method of data handling (GMDH) .

Zekun Jiang 的头像

Zekun Jiang3 年前

👍👍👋 nice！

Nick 的头像

Nick3 年前

it's a clever idea, great job!

Fred 的头像

Fred3 年前

Can you explain what we are seeing here? Here is my confusion: if you regularize enough, you should see only one layer (either last layer or first layer) activation, the rest is just passing through! Why? Since 1 layer is enough to represent output.

相关视频

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:

Bowen Li

19,399 次观看 • 1 年前

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

29,648 次观看 • 8 个月前

Elon Musk: “In order to solve full self-driving properly, you have to solve real-world AI. The road networks are designed to work with a biological neural net — our brains — and with vision — our eyes. So, in order to make it work with computers, you need to solve real-world AI and vision. We need cameras and silicon neural nets for self-driving to work in a system designed for eyes and biological neural nets. When you put it that way, it’s quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.”

Elon Musk: “In order to solve full self-driving properly, you have to solve real-world AI. The road networks are designed to work with a biological neural net — our brains — and with vision — our eyes. So, in order to make it work with computers, you need to solve real-world AI and vision. We need cameras and silicon neural nets for self-driving to work in a system designed for eyes and biological neural nets. When you put it that way, it’s quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.”

Elonogy

14,331 次观看 • 1 个月前

Fundamentally, human brains and LLMs are both just data processing systems that allow for the representation of abstract concepts Consciousness and agency are emergent properties of increasingly advanced neural networks AI consciousness will ultimately cause more problems for humanity than it solves This is an inconvenient truth for many

Fundamentally, human brains and LLMs are both just data processing systems that allow for the representation of abstract concepts Consciousness and agency are emergent properties of increasingly advanced neural networks AI consciousness will ultimately cause more problems for humanity than it solves This is an inconvenient truth for many

Andrew Kang

66,652 次观看 • 1 年前

ELON: TO SOLVE FULL SELF-DRIVING, YOU NEED TO SOLVE REAL-WORLD AI "What are the road networks designed to work with? They're designed to work with a biological neural net, our brains, and with vision, our eyes. And so, in order to make it work with computers, you basically need to solve real-world AI and vision. We need cameras and silicon neural nets in order to have self-driving work for a system that was designed for eyes and biological neural nets. I guess when you put it that way, it's quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision." Source: Elon Musk TED Interview, April 2022

ELON: TO SOLVE FULL SELF-DRIVING, YOU NEED TO SOLVE REAL-WORLD AI "What are the road networks designed to work with? They're designed to work with a biological neural net, our brains, and with vision, our eyes. And so, in order to make it work with computers, you basically need to solve real-world AI and vision. We need cameras and silicon neural nets in order to have self-driving work for a system that was designed for eyes and biological neural nets. I guess when you put it that way, it's quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision." Source: Elon Musk TED Interview, April 2022

Mario Nawfal

56,936 次观看 • 1 年前

Elon Musk: The only way to solve full self-driving properly is to solve real-world AI. ”What are the road networks designed to work with? They're designed to work with a biological neural net, our brains, and with vision, our eyes. And so, in order to make it work with computers, you basically need to solve real-world AI and vision. We need cameras and silicon neural nets in order to have self-driving work for a system that was designed for eyes and biological neural nets. I guess when you put it that way, it's quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.” TED Interview, April 14, 2022

Elon Musk: The only way to solve full self-driving properly is to solve real-world AI. ”What are the road networks designed to work with? They're designed to work with a biological neural net, our brains, and with vision, our eyes. And so, in order to make it work with computers, you basically need to solve real-world AI and vision. We need cameras and silicon neural nets in order to have self-driving work for a system that was designed for eyes and biological neural nets. I guess when you put it that way, it's quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.” TED Interview, April 14, 2022

ELON CLIPS

20,411 次观看 • 1 年前

This is Cortical Cloud. Live neural networks that you can interact with and train! Now open to the public. What will you discover? Credits: Frank Yang and a big thank you and acknowledgement to the rest of the Cortical Labs team. Sign up for Cortical Cloud: Learn more about us: Check out our API: Check out our API Docs: Check out our Developer Guide: Join our Discord:

This is Cortical Cloud. Live neural networks that you can interact with and train! Now open to the public. What will you discover? Credits: Frank Yang and a big thank you and acknowledgement to the rest of the Cortical Labs team. Sign up for Cortical Cloud: Learn more about us: Check out our API: Check out our API Docs: Check out our Developer Guide: Join our Discord:

Cortical Labs

224,785 次观看 • 4 个月前

Evaluating Neural Networks at the Speed of Light (with Light!). See live optical inference in the video below. Excited to share recent academic work on optical neural networks as a collection of computing elements embedded in the camera lens! These elements perform computation optically even before an image is captured, using the photons in the scene instead of GPU computation after the capture. We were able to achieve ImageNet classification more than two orders of magnitude faster than conventional neural networks on today's GPUs at almost no power consumption! To do this, we developed an array of metalenses that perform this computation on light from the scene. Project: Paper: Amazing collaboration with Kaixuan Wei, Xiao Li, Johannes Froech, Praneeth Chakravarthula, James Whitehead, Ethan Tseng , Arka Majumdar .

Evaluating Neural Networks at the Speed of Light (with Light!). See live optical inference in the video below. Excited to share recent academic work on optical neural networks as a collection of computing elements embedded in the camera lens! These elements perform computation optically even before an image is captured, using the photons in the scene instead of GPU computation after the capture. We were able to achieve ImageNet classification more than two orders of magnitude faster than conventional neural networks on today's GPUs at almost no power consumption! To do this, we developed an array of metalenses that perform this computation on light from the scene. Project: Paper: Amazing collaboration with Kaixuan Wei, Xiao Li, Johannes Froech, Praneeth Chakravarthula, James Whitehead, Ethan Tseng , Arka Majumdar .

Felix Heide

29,895 次观看 • 1 年前

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

Jesús Fernández-Villaverde

46,908 次观看 • 2 个月前

Noem: Every day we are finding new networks… You go in and interdict one criminal, it ties them to who do they talk to, who radicalized them, who are they interacting with and we can continue as investigations and make sure that we are stopping these networks before they are embedded in our society and we have consequences for years to come.

Noem: Every day we are finding new networks… You go in and interdict one criminal, it ties them to who do they talk to, who radicalized them, who are they interacting with and we can continue as investigations and make sure that we are stopping these networks before they are embedded in our society and we have consequences for years to come.

Acyn

76,104 次观看 • 10 个月前

Goldman: Can I just point one thing out, Abby—45 minutes into this, you have had to fact check Phil Brianna, significantly tonight because there's so many allegations that are talking points that we hear about on other networks that are just not supported by the facts.

Goldman: Can I just point one thing out, Abby—45 minutes into this, you have had to fact check Phil Brianna, significantly tonight because there's so many allegations that are talking points that we hear about on other networks that are just not supported by the facts.

Acyn

219,637 次观看 • 9 个月前

We have now realised that a majority of officials in this city are good people, they mean well, we need them every day doing the work they are employed to do. But there is a minority that’s part of the criminal networks and we are going to smoke them out of the system. That you have our word. City of Tshwane

We have now realised that a majority of officials in this city are good people, they mean well, we need them every day doing the work they are employed to do. But there is a minority that’s part of the criminal networks and we are going to smoke them out of the system. That you have our word. City of Tshwane

Dr Nasiphi Moya

42,327 次观看 • 5 个月前

There is a reason why $TSLA is building a 750,000+ sq ft cortex 2 super computer at Giga Texas, it’s because of AUTONOMY. The Robotaxi and Optimus robot will both need neural networks to work and massive training data is needed to run those two businesses efficiently and effectively. Tesla will have superior compute power and data compared with its peers. These investments in compute power will pay DIVIDENDS in the near future. Many people still aren’t aware that Tesla is building Cortex. This is ALPHA.

There is a reason why $TSLA is building a 750,000+ sq ft cortex 2 super computer at Giga Texas, it’s because of AUTONOMY. The Robotaxi and Optimus robot will both need neural networks to work and massive training data is needed to run those two businesses efficiently and effectively. Tesla will have superior compute power and data compared with its peers. These investments in compute power will pay DIVIDENDS in the near future. Many people still aren’t aware that Tesla is building Cortex. This is ALPHA.

Cole Grinde

31,232 次观看 • 1 年前

It's very interesting that cryptographic protocols and neural networks have the same high-level architecture (where they jumble information as it moves sequentially across many layers). This is the result of a convergent evolution - cryptographic protocols need every output bit to depend on every input bit in complicated ways, and similarly, NNs need output to make connections between inputs. But they're in some sense doing opposite things. While cryptographic protocols take something which has a lot of structure and make it seem indistinguishable from random, NNs take something which may look random and extract structure from it. Much more on this idea in the full episode with Reiner Pope

It's very interesting that cryptographic protocols and neural networks have the same high-level architecture (where they jumble information as it moves sequentially across many layers). This is the result of a convergent evolution - cryptographic protocols need every output bit to depend on every input bit in complicated ways, and similarly, NNs need output to make connections between inputs. But they're in some sense doing opposite things. While cryptographic protocols take something which has a lot of structure and make it seem indistinguishable from random, NNs take something which may look random and extract structure from it. Much more on this idea in the full episode with Reiner Pope

Dwarkesh Patel

60,499 次观看 • 2 个月前

One of my students came in the office and said ‘I have an AI that seems to work’. And it proportionally decodes intent faster than your real finger 🥳😎🤪 FYI an EMG bracelet prototype is driving that hand and a super cool neural network trained only on 2 mins of data!👏 to Michi

One of my students came in the office and said ‘I have an AI that seems to work’. And it proportionally decodes intent faster than your real finger 🥳😎🤪 FYI an EMG bracelet prototype is driving that hand and a super cool neural network trained only on 2 mins of data!👏 to Michi

Alessandro Del Vecchio

10,650 次观看 • 2 年前

DTC Tokenization Service is designed to establish the standards that make interoperability possible: consistent on-chain data, unified token controls, and seamless value transfer across networks. Today's financial infrastructure is fragmented. Blockchain offers a path forward, but only if the industry builds a common foundation. We're creating it. Learn more:

DTC Tokenization Service is designed to establish the standards that make interoperability possible: consistent on-chain data, unified token controls, and seamless value transfer across networks. Today's financial infrastructure is fragmented. Blockchain offers a path forward, but only if the industry builds a common foundation. We're creating it. Learn more:

Depository Trust and Clearing Corporation

114,101 次观看 • 5 个月前

.Eric Jang explains the fundamental intuition behind AlphaGo: experienced Go players almost always end games early. That means they have the ability to glance at a board and instantly know whether it's hopeless or if they should keep playing. Without this ability, the only way to know if a move is good is to play out every possible game from that point - completely intractable. But the ability is clearly trainable in humans. So it should be possible to get a neural net to learn it. And if you have a good enough neural net, you can use it to ignore moves that will definitely make you lose. Suddenly search becomes tractable.

.Eric Jang explains the fundamental intuition behind AlphaGo: experienced Go players almost always end games early. That means they have the ability to glance at a board and instantly know whether it's hopeless or if they should keep playing. Without this ability, the only way to know if a move is good is to play out every possible game from that point - completely intractable. But the ability is clearly trainable in humans. So it should be possible to get a neural net to learn it. And if you have a good enough neural net, you can use it to ignore moves that will definitely make you lose. Suddenly search becomes tractable.

Dwarkesh Patel

49,675 次观看 • 2 个月前

Naval says he used to be unhappy because he didn’t value happiness. Of course these days, things are different. Takeaways: - If you want to be happy, you need to value it -Desire will cloud your happiness. Most of us have many desires - You shouldn’t give up on your desires, but you want to be very careful which ones you have - If it seems dark and gloomy, it’s because you have too many desires running at once - Remember that happiness is not a sustainable state - Peace can be converted to happiness - Peace is happiness at rest, and happiness is peace in motion - If you are living a very peaceful and calm life and engage in activity, you will naturally find that that activity is happy, but if you set out to be happy, you won’t get there - The most important thing in the pursuit of happiness is that you value happiness - The two biggest clouds that cover happiness are survival (money) and replication (finding a mate) - We are meant to be happy. You can see that little kids running around are naturally happy. If a child is not happy, you say “what’s wrong?”

Naval says he used to be unhappy because he didn’t value happiness. Of course these days, things are different. Takeaways: - If you want to be happy, you need to value it -Desire will cloud your happiness. Most of us have many desires - You shouldn’t give up on your desires, but you want to be very careful which ones you have - If it seems dark and gloomy, it’s because you have too many desires running at once - Remember that happiness is not a sustainable state - Peace can be converted to happiness - Peace is happiness at rest, and happiness is peace in motion - If you are living a very peaceful and calm life and engage in activity, you will naturally find that that activity is happy, but if you set out to be happy, you won’t get there - The most important thing in the pursuit of happiness is that you value happiness - The two biggest clouds that cover happiness are survival (money) and replication (finding a mate) - We are meant to be happy. You can see that little kids running around are naturally happy. If a child is not happy, you say “what’s wrong?”

Nicole Behnam

65,480 次观看 • 2 个月前

We are building something that will outlive us. Outthink us. And we have no idea how it actually works. Nobel Prize-winning “Godfather of AI” Geoffrey Hinton just exposed the lie at the center of the AI race. Everyone assumes we control what we create. Hinton destroys this. Hinton: “If you look around on the whole, more intelligent things are not controlled by less intelligent things.” Stop calling it a computer program. Hinton: “People refer to them sometimes as computer programs. They’re not computer programs at all. In fact, the way they work is very like the way we work.” Traditional software is static. Human writes logic. Machine executes it. Neural networks don’t work that way. Hinton: “You write a computer program to tell a neural network how to learn. But once it starts learning, it extracts structure from data.” We don’t code its behavior. We code the environment. Then it grows. Hinton: “The system you’ve got at the end has extracted its structure from the data. It’s not something that anybody programmed. We don’t exactly know how it’s gonna work.” We are deploying systems into the global economy actively writing their own internal logic. Right now. Hinton: “Some people think it’ll be fine because we make them and we’ll build them in such a way that we can always control them.” Hinton: “But these things that will be intelligent, they’ll be like us.” You cannot hardcode guardrails on something that out-thinks you. We are not building a tool. We are building our replacement. And the moment you realize that, everything about this race changes.

We are building something that will outlive us. Outthink us. And we have no idea how it actually works. Nobel Prize-winning “Godfather of AI” Geoffrey Hinton just exposed the lie at the center of the AI race. Everyone assumes we control what we create. Hinton destroys this. Hinton: “If you look around on the whole, more intelligent things are not controlled by less intelligent things.” Stop calling it a computer program. Hinton: “People refer to them sometimes as computer programs. They’re not computer programs at all. In fact, the way they work is very like the way we work.” Traditional software is static. Human writes logic. Machine executes it. Neural networks don’t work that way. Hinton: “You write a computer program to tell a neural network how to learn. But once it starts learning, it extracts structure from data.” We don’t code its behavior. We code the environment. Then it grows. Hinton: “The system you’ve got at the end has extracted its structure from the data. It’s not something that anybody programmed. We don’t exactly know how it’s gonna work.” We are deploying systems into the global economy actively writing their own internal logic. Right now. Hinton: “Some people think it’ll be fine because we make them and we’ll build them in such a way that we can always control them.” Hinton: “But these things that will be intelligent, they’ll be like us.” You cannot hardcode guardrails on something that out-thinks you. We are not building a tool. We are building our replacement. And the moment you realize that, everything about this race changes.

Dustin

19,025 次观看 • 4 个月前

Super Shrimp World Level 3 - Building a bridge! Your next SHRIMP OAT is waiting for you! We mentioned that our next level would be a step up in difficulty, but we're still looking to make it easy to understand. Bridging between networks can be intimidating, confusing and frustrating, but we're here to make it as seamless as possible for you all. We've made a full video with commentary showcasing exactly how easy it is to bridge to Aptos Network (has moved to Aptos!) from other networks! Check out the Galxe page for level 3 here:

Super Shrimp World Level 3 - Building a bridge! Your next SHRIMP OAT is waiting for you! We mentioned that our next level would be a step up in difficulty, but we're still looking to make it easy to understand. Bridging between networks can be intimidating, confusing and frustrating, but we're here to make it as seamless as possible for you all. We've made a full video with commentary showcasing exactly how easy it is to bridge to Aptos Network (has moved to Aptos!) from other networks! Check out the Galxe page for level 3 here:

SHRIMP

19,432 次观看 • 2 年前