Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Many scientific problems hinge on finding interpretable formulas that fit data, but neural networks are the outright opposite! Check out our recent work that make neural networks modular and interpretable. If you have interesting datasets at hand, we're happy to collaborate!

Ziming Liu

14,295 subscribers

62,120 Aufrufe • vor 3 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Ziming Liu

Ziming Liuvor 3 Jahren

paper: code:

Profilbild von Ziming Liu

Ziming Liuvor 3 Jahren

Symbolic regression (finding symbolic formulas from data) has been quite successful on some examples, but it also fail completely for others. Instead, our goal is structure regression (finding modular structures from data), which can provide visible insights for all cases.

Profilbild von Lei Chen

Lei Chenvor 3 Jahren

Interesting work! Is this doable for graph nets? The task would be to locate significant attributed substructures. (I guess such location might be useful to help predesign the library of GSN @mmbronstein )

Profilbild von ExiledInfoHaz

ExiledInfoHazvor 3 Jahren

👀

Profilbild von Mitchell B. Slapik

Mitchell B. Slapikvor 3 Jahren

Great work Ziming! Could you explain Figure 14? I was a bit confused by it.

Profilbild von Sam McKenzie

Sam McKenzievor 3 Jahren

We suffer in the seizure forecasting field making interpretable models because the features extracted from neural time series are highly correlated. Any tool to understand why our models work would give important insights as to how the brain changes in the lead up to a seizure.

Profilbild von bobert

bobertvor 3 Jahren

I see some resemblance with Group method of data handling (GMDH) .

Profilbild von Zekun Jiang

Zekun Jiangvor 3 Jahren

👍👍👋 nice！

Profilbild von Nick

Nickvor 3 Jahren

it's a clever idea, great job!

Profilbild von Fred

Fredvor 3 Jahren

Can you explain what we are seeing here? Here is my confusion: if you regularize enough, you should see only one layer (either last layer or first layer) activation, the rest is just passing through! Why? Since 1 layer is enough to represent output.

Ähnliche Videos

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:

Bowen Li

19,495 Aufrufe • vor 1 Jahr

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Ever wondered what neural networks are and how they work? Systems like ChatGPT use neural networks to work as well as they do. Neural networks are composed of "layers" of neurons, layers with different functions; connections between layers called "weights"; and mathematical functions called "activation functions". If you’re interested in learning about these systems, check the comments. Ultimately, the neural network structure of the model serves to visually demonstrate that it is, in fact, a complex mathematical equation. When companies release the model's weights, they are releasing a key component needed to run the model's complete equation. Without the weights, the equation is incomplete. For the math-minded: the weights of a model are the learned numbers (they are variables during training) that are then used as constants in the mathematical functions that make up the model. Neural networks are ultimately just one big, hyper-complex mathematical function, and when a model is trained, it learns the constants associated with the high-dimensional input.

Harper Carroll

37,897 Aufrufe • vor 10 Monaten

Fundamentally, human brains and LLMs are both just data processing systems that allow for the representation of abstract concepts Consciousness and agency are emergent properties of increasingly advanced neural networks AI consciousness will ultimately cause more problems for humanity than it solves This is an inconvenient truth for many

Fundamentally, human brains and LLMs are both just data processing systems that allow for the representation of abstract concepts Consciousness and agency are emergent properties of increasingly advanced neural networks AI consciousness will ultimately cause more problems for humanity than it solves This is an inconvenient truth for many

Andrew Kang

66,652 Aufrufe • vor 1 Jahr

This is Cortical Cloud. Live neural networks that you can interact with and train! Now open to the public. What will you discover? Credits: Frank Yang and a big thank you and acknowledgement to the rest of the Cortical Labs team. Sign up for Cortical Cloud: Learn more about us: Check out our API: Check out our API Docs: Check out our Developer Guide: Join our Discord:

This is Cortical Cloud. Live neural networks that you can interact with and train! Now open to the public. What will you discover? Credits: Frank Yang and a big thank you and acknowledgement to the rest of the Cortical Labs team. Sign up for Cortical Cloud: Learn more about us: Check out our API: Check out our API Docs: Check out our Developer Guide: Join our Discord:

Cortical Labs

228,547 Aufrufe • vor 5 Monaten

Evaluating Neural Networks at the Speed of Light (with Light!). See live optical inference in the video below. Excited to share recent academic work on optical neural networks as a collection of computing elements embedded in the camera lens! These elements perform computation optically even before an image is captured, using the photons in the scene instead of GPU computation after the capture. We were able to achieve ImageNet classification more than two orders of magnitude faster than conventional neural networks on today's GPUs at almost no power consumption! To do this, we developed an array of metalenses that perform this computation on light from the scene. Project: Paper: Amazing collaboration with Kaixuan Wei, Xiao Li, Johannes Froech, Praneeth Chakravarthula, James Whitehead, Ethan Tseng , Arka Majumdar .

Evaluating Neural Networks at the Speed of Light (with Light!). See live optical inference in the video below. Excited to share recent academic work on optical neural networks as a collection of computing elements embedded in the camera lens! These elements perform computation optically even before an image is captured, using the photons in the scene instead of GPU computation after the capture. We were able to achieve ImageNet classification more than two orders of magnitude faster than conventional neural networks on today's GPUs at almost no power consumption! To do this, we developed an array of metalenses that perform this computation on light from the scene. Project: Paper: Amazing collaboration with Kaixuan Wei, Xiao Li, Johannes Froech, Praneeth Chakravarthula, James Whitehead, Ethan Tseng , Arka Majumdar .

Felix Heide

29,904 Aufrufe • vor 1 Jahr

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

Jesús Fernández-Villaverde

47,212 Aufrufe • vor 4 Monaten

We have now realised that a majority of officials in this city are good people, they mean well, we need them every day doing the work they are employed to do. But there is a minority that’s part of the criminal networks and we are going to smoke them out of the system. That you have our word. City of Tshwane

We have now realised that a majority of officials in this city are good people, they mean well, we need them every day doing the work they are employed to do. But there is a minority that’s part of the criminal networks and we are going to smoke them out of the system. That you have our word. City of Tshwane

Dr Nasiphi Moya

42,355 Aufrufe • vor 7 Monaten

It's very interesting that cryptographic protocols and neural networks have the same high-level architecture (where they jumble information as it moves sequentially across many layers). This is the result of a convergent evolution - cryptographic protocols need every output bit to depend on every input bit in complicated ways, and similarly, NNs need output to make connections between inputs. But they're in some sense doing opposite things. While cryptographic protocols take something which has a lot of structure and make it seem indistinguishable from random, NNs take something which may look random and extract structure from it. Much more on this idea in the full episode with Reiner Pope

It's very interesting that cryptographic protocols and neural networks have the same high-level architecture (where they jumble information as it moves sequentially across many layers). This is the result of a convergent evolution - cryptographic protocols need every output bit to depend on every input bit in complicated ways, and similarly, NNs need output to make connections between inputs. But they're in some sense doing opposite things. While cryptographic protocols take something which has a lot of structure and make it seem indistinguishable from random, NNs take something which may look random and extract structure from it. Much more on this idea in the full episode with Reiner Pope

Dwarkesh Patel

60,640 Aufrufe • vor 4 Monaten

One of my students came in the office and said ‘I have an AI that seems to work’. And it proportionally decodes intent faster than your real finger 🥳😎🤪 FYI an EMG bracelet prototype is driving that hand and a super cool neural network trained only on 2 mins of data!👏 to Michi

One of my students came in the office and said ‘I have an AI that seems to work’. And it proportionally decodes intent faster than your real finger 🥳😎🤪 FYI an EMG bracelet prototype is driving that hand and a super cool neural network trained only on 2 mins of data!👏 to Michi

Alessandro Del Vecchio

10,652 Aufrufe • vor 2 Jahren

DTC Tokenization Service is designed to establish the standards that make interoperability possible: consistent on-chain data, unified token controls, and seamless value transfer across networks. Today's financial infrastructure is fragmented. Blockchain offers a path forward, but only if the industry builds a common foundation. We're creating it. Learn more:

DTC Tokenization Service is designed to establish the standards that make interoperability possible: consistent on-chain data, unified token controls, and seamless value transfer across networks. Today's financial infrastructure is fragmented. Blockchain offers a path forward, but only if the industry builds a common foundation. We're creating it. Learn more:

Depository Trust and Clearing Corporation

114,101 Aufrufe • vor 7 Monaten

The odds are often against you in exploration, but every geologist shares the same goal: to make a discovery. At $KFR.V, we're striving for more than just finding mineralization. We're working toward discoveries with the potential to stand the test of time, create lasting value, support communities, and leave a positive legacy for generations. "If you have a good project, a good team, and you have access to capital, that's the exact formula that leads to making multiple discoveries and bucking that trend." - CEO Dustin Perry ⛰️ Check out our new video and discover what drives the pursuit. ⬇️

The odds are often against you in exploration, but every geologist shares the same goal: to make a discovery. At $KFR.V, we're striving for more than just finding mineralization. We're working toward discoveries with the potential to stand the test of time, create lasting value, support communities, and leave a positive legacy for generations. "If you have a good project, a good team, and you have access to capital, that's the exact formula that leads to making multiple discoveries and bucking that trend." - CEO Dustin Perry ⛰️ Check out our new video and discover what drives the pursuit. ⬇️

Kingfisher Metals Corp.

2,448,991 Aufrufe • vor 2 Monaten

.Eric Jang explains the fundamental intuition behind AlphaGo: experienced Go players almost always end games early. That means they have the ability to glance at a board and instantly know whether it's hopeless or if they should keep playing. Without this ability, the only way to know if a move is good is to play out every possible game from that point - completely intractable. But the ability is clearly trainable in humans. So it should be possible to get a neural net to learn it. And if you have a good enough neural net, you can use it to ignore moves that will definitely make you lose. Suddenly search becomes tractable.

.Eric Jang explains the fundamental intuition behind AlphaGo: experienced Go players almost always end games early. That means they have the ability to glance at a board and instantly know whether it's hopeless or if they should keep playing. Without this ability, the only way to know if a move is good is to play out every possible game from that point - completely intractable. But the ability is clearly trainable in humans. So it should be possible to get a neural net to learn it. And if you have a good enough neural net, you can use it to ignore moves that will definitely make you lose. Suddenly search becomes tractable.

Dwarkesh Patel

49,692 Aufrufe • vor 3 Monaten

Naval says he used to be unhappy because he didn’t value happiness. Of course these days, things are different. Takeaways: - If you want to be happy, you need to value it -Desire will cloud your happiness. Most of us have many desires - You shouldn’t give up on your desires, but you want to be very careful which ones you have - If it seems dark and gloomy, it’s because you have too many desires running at once - Remember that happiness is not a sustainable state - Peace can be converted to happiness - Peace is happiness at rest, and happiness is peace in motion - If you are living a very peaceful and calm life and engage in activity, you will naturally find that that activity is happy, but if you set out to be happy, you won’t get there - The most important thing in the pursuit of happiness is that you value happiness - The two biggest clouds that cover happiness are survival (money) and replication (finding a mate) - We are meant to be happy. You can see that little kids running around are naturally happy. If a child is not happy, you say “what’s wrong?”

Naval says he used to be unhappy because he didn’t value happiness. Of course these days, things are different. Takeaways: - If you want to be happy, you need to value it -Desire will cloud your happiness. Most of us have many desires - You shouldn’t give up on your desires, but you want to be very careful which ones you have - If it seems dark and gloomy, it’s because you have too many desires running at once - Remember that happiness is not a sustainable state - Peace can be converted to happiness - Peace is happiness at rest, and happiness is peace in motion - If you are living a very peaceful and calm life and engage in activity, you will naturally find that that activity is happy, but if you set out to be happy, you won’t get there - The most important thing in the pursuit of happiness is that you value happiness - The two biggest clouds that cover happiness are survival (money) and replication (finding a mate) - We are meant to be happy. You can see that little kids running around are naturally happy. If a child is not happy, you say “what’s wrong?”

Nicole Behnam

65,600 Aufrufe • vor 4 Monaten

We are building something that will outlive us. Outthink us. And we have no idea how it actually works. Nobel Prize-winning “Godfather of AI” Geoffrey Hinton just exposed the lie at the center of the AI race. Everyone assumes we control what we create. Hinton destroys this. Hinton: “If you look around on the whole, more intelligent things are not controlled by less intelligent things.” Stop calling it a computer program. Hinton: “People refer to them sometimes as computer programs. They’re not computer programs at all. In fact, the way they work is very like the way we work.” Traditional software is static. Human writes logic. Machine executes it. Neural networks don’t work that way. Hinton: “You write a computer program to tell a neural network how to learn. But once it starts learning, it extracts structure from data.” We don’t code its behavior. We code the environment. Then it grows. Hinton: “The system you’ve got at the end has extracted its structure from the data. It’s not something that anybody programmed. We don’t exactly know how it’s gonna work.” We are deploying systems into the global economy actively writing their own internal logic. Right now. Hinton: “Some people think it’ll be fine because we make them and we’ll build them in such a way that we can always control them.” Hinton: “But these things that will be intelligent, they’ll be like us.” You cannot hardcode guardrails on something that out-thinks you. We are not building a tool. We are building our replacement. And the moment you realize that, everything about this race changes.

We are building something that will outlive us. Outthink us. And we have no idea how it actually works. Nobel Prize-winning “Godfather of AI” Geoffrey Hinton just exposed the lie at the center of the AI race. Everyone assumes we control what we create. Hinton destroys this. Hinton: “If you look around on the whole, more intelligent things are not controlled by less intelligent things.” Stop calling it a computer program. Hinton: “People refer to them sometimes as computer programs. They’re not computer programs at all. In fact, the way they work is very like the way we work.” Traditional software is static. Human writes logic. Machine executes it. Neural networks don’t work that way. Hinton: “You write a computer program to tell a neural network how to learn. But once it starts learning, it extracts structure from data.” We don’t code its behavior. We code the environment. Then it grows. Hinton: “The system you’ve got at the end has extracted its structure from the data. It’s not something that anybody programmed. We don’t exactly know how it’s gonna work.” We are deploying systems into the global economy actively writing their own internal logic. Right now. Hinton: “Some people think it’ll be fine because we make them and we’ll build them in such a way that we can always control them.” Hinton: “But these things that will be intelligent, they’ll be like us.” You cannot hardcode guardrails on something that out-thinks you. We are not building a tool. We are building our replacement. And the moment you realize that, everything about this race changes.

Dustin

19,043 Aufrufe • vor 6 Monaten

Super Shrimp World Level 3 - Building a bridge! Your next SHRIMP OAT is waiting for you! We mentioned that our next level would be a step up in difficulty, but we're still looking to make it easy to understand. Bridging between networks can be intimidating, confusing and frustrating, but we're here to make it as seamless as possible for you all. We've made a full video with commentary showcasing exactly how easy it is to bridge to Aptos Network (has moved to Aptos!) from other networks! Check out the Galxe page for level 3 here:

Super Shrimp World Level 3 - Building a bridge! Your next SHRIMP OAT is waiting for you! We mentioned that our next level would be a step up in difficulty, but we're still looking to make it easy to understand. Bridging between networks can be intimidating, confusing and frustrating, but we're here to make it as seamless as possible for you all. We've made a full video with commentary showcasing exactly how easy it is to bridge to Aptos Network (has moved to Aptos!) from other networks! Check out the Galxe page for level 3 here:

SHRIMP

19,445 Aufrufe • vor 2 Jahren