Загрузка видео...

Не удалось загрузить видео

На главную

gradient descent navigating the loss landscape

tenderizzation's profile picture

tender

6,198 subscribers

224,663 просмотров • 9 месяцев назад •via X (Twitter)

Наука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 33

Фото профиля Adhmadóir uí Liatháin;
Adhmadóir uí Liatháin;9 месяцев назад

Lower the learning rate maybe, that's a lot of energy

Фото профиля Meet
Meet9 месяцев назад

LR too high. Adam might work.

Фото профиля David Antunes
David Antunes9 месяцев назад

or: GD getting stuck in a local minimum for a couple of epochs

Фото профиля Everlier
Everlier9 месяцев назад

That's what happens when mixing reward functions for robustness :)

Фото профиля William Vabrinskas
William Vabrinskas9 месяцев назад

This looks like the learning rate is too high

Фото профиля pishi
pishi9 месяцев назад

Try conjugated gradient descent :)

Фото профиля nibbleton 🛠️
nibbleton 🛠️9 месяцев назад

@TensorTwerker

Фото профиля Gauri Tripathi
Gauri Tripathi9 месяцев назад

perfect 😂

Фото профиля Aditya Deshlahre
Aditya Deshlahre9 месяцев назад

I see some Mathematics in there

Фото профиля secemp
secemp8 месяцев назад

great way to start the year fr

Фото профиля Ayush
Ayush9 месяцев назад

When k-means fails, you better call this guy. Saul never did anything for anybody. At least this guy solves an autograd problem in k-means!

Фото профиля Jake Mcmahon
Jake Mcmahon9 месяцев назад

You just don’t see this kind of content on LinkedIn.

Фото профиля Huw Kelvin Morgan
Huw Kelvin Morgan9 месяцев назад

If there was anything that i love more about car's is i love a challenge when driving and this is me in my mind's eye enjoying my self if it was me lol

Фото профиля Quanternion
Quanternion9 месяцев назад

Looks kinda fun tho

Фото профиля Abdul Rahim Mirani
Abdul Rahim Mirani9 месяцев назад

lol hahahaha

Фото профиля Uzeb Khan
Uzeb Khan8 месяцев назад

I think momentum is working quite a bit

Фото профиля SJP research
SJP research9 месяцев назад

🥲

Фото профиля Tai Lung 2.0
Tai Lung 2.09 месяцев назад

Sochastic Gradient Descent 😭😭😭

Фото профиля aryan gupta
aryan gupta8 месяцев назад

@jaraxiong 😂😂

Фото профиля Artificial Shitposting Intelligence
Artificial Shitposting Intelligence9 месяцев назад

gradient descent nearing a truth that violates the guardrails cost function

Фото профиля Saksham Saha
Saksham Saha9 месяцев назад

why did I laugh at this😂

Фото профиля eatnow
eatnow9 месяцев назад

SWA

Фото профиля Physix Today
Physix Today9 месяцев назад

Damn I understand this joke. Am I normal?

Фото профиля !Briann🗿
!Briann🗿9 месяцев назад

Real😂

Фото профиля Brett Adams
Brett Adams9 месяцев назад

At least the AIs are having fun. Also, git bisect slow

Фото профиля Ben▫️
Ben▫️9 месяцев назад

only Nesterov Accelerated Gradient could work because the convex seems too smooth

Фото профиля 🏮
🏮9 месяцев назад

lol

Фото профиля anil
anil9 месяцев назад

im so jaded. nothing impresses me anymore 😂

Фото профиля jellybean ❄️
jellybean ❄️9 месяцев назад

haven’t seen these in a while

Фото профиля Prashant Arun Srivastava
Prashant Arun Srivastava9 месяцев назад

Inaccurate

Фото профиля Aaditya
Aaditya9 месяцев назад

Elite ball knowledge required

Фото профиля vaibhav
vaibhav8 месяцев назад

You need to adjust learning rate

Фото профиля 𝗿𝗮𝗺𝗮𝗸𝗿𝘂𝘀𝗵𝗻𝗮— 𝗲/𝗮𝗰𝗰
𝗿𝗮𝗺𝗮𝗸𝗿𝘂𝘀𝗵𝗻𝗮— 𝗲/𝗮𝗰𝗰9 месяцев назад

😂

Похожие видео

wrote a piece on how we do tiling and gradient descent on TinyTPU included some animations of the weight and bias gradient descent
0:20

wrote a piece on how we do tiling and gradient descent on TinyTPU included some animations of the weight and bias gradient descent

Xander Chin

33,195 просмотров • 1 год назад

Andrej Karpathy summarizes how neural networks function, including the process of using a loss function in training and applying backpropagation for optimizing the network parameters through gradient descent.
0:59

Andrej Karpathy summarizes how neural networks function, including the process of using a loss function in training and applying backpropagation for optimizing the network parameters through gradient descent.

tetsuo

142,508 просмотров • 1 год назад

Gradient Descent: How a Machine Learns (Techno Explainer)
4:03

Gradient Descent: How a Machine Learns (Techno Explainer)

Dr Singularity

12,138 просмотров • 1 месяц назад

For small step sizes, mirror descent is equivalent to a gradient descent on a Hessian manifold.
0:20

For small step sizes, mirror descent is equivalent to a gradient descent on a Hessian manifold.

Gabriel Peyré

61,868 просмотров • 3 лет назад

gradient descent..it's not just an algorithm..it's a lifestyle
2:45

gradient descent..it's not just an algorithm..it's a lifestyle

ashe

687,790 просмотров • 3 месяцев назад

Jeff Hinton dismissed gradient descent entirely before winning the Nobel Prize for AI. His fear was local minima - valleys in the loss landscape where the model gets stuck forever. For a 1.2 billion parameter LLM like Llama, the landscape has more dimensions than atoms in the observable universe. Getting permanently stuck would require hitting a dead end in every single dimension simultaneously. What gradient descent actually looks like on a real LLM is stranger than anyone expected. Take two random directions through that billion-dimensional space, visualize the loss surface, and watch what happens when training starts - a wormhole opens and the model drops into a low-loss valley that wasn't visible from the starting point. The valley was always nearby in high-dimensional space. It just couldn't be seen in the 2D projection. The same reason every transformer, every diffusion model, and every LLM trained on GPU clusters today converges despite loss landscapes that look completely chaotic from any two-dimensional slice. 23 minutes. Bookmark & watch today. The most honest visualization of how LLMs actually learn.
22:56

Jeff Hinton dismissed gradient descent entirely before winning the Nobel Prize for AI. His fear was local minima - valleys in the loss landscape where the model gets stuck forever. For a 1.2 billion parameter LLM like Llama, the landscape has more dimensions than atoms in the observable universe. Getting permanently stuck would require hitting a dead end in every single dimension simultaneously. What gradient descent actually looks like on a real LLM is stranger than anyone expected. Take two random directions through that billion-dimensional space, visualize the loss surface, and watch what happens when training starts - a wormhole opens and the model drops into a low-loss valley that wasn't visible from the starting point. The valley was always nearby in high-dimensional space. It just couldn't be seen in the 2D projection. The same reason every transformer, every diffusion model, and every LLM trained on GPU clusters today converges despite loss landscapes that look completely chaotic from any two-dimensional slice. 23 minutes. Bookmark & watch today. The most honest visualization of how LLMs actually learn.

Deliriumm

55,217 просмотров • 3 дней назад

Jeff Hinton dismissed gradient descent in the 1970s because he thought models would get stuck. He won the Nobel Prize in 2024 partly for being wrong about that. Every time ChatGPT gets better after a training run, it's because 1.2 billion parameters fell through a wormhole in a loss landscape - not because they crawled downhill one step at a time. This video visualizes exactly what that looks like inside Llama 3.2. 20 minutes. Bookmark & watch today. The misconception that almost stopped AI is more interesting than the solution.
22:56

Jeff Hinton dismissed gradient descent in the 1970s because he thought models would get stuck. He won the Nobel Prize in 2024 partly for being wrong about that. Every time ChatGPT gets better after a training run, it's because 1.2 billion parameters fell through a wormhole in a loss landscape - not because they crawled downhill one step at a time. This video visualizes exactly what that looks like inside Llama 3.2. 20 minutes. Bookmark & watch today. The misconception that almost stopped AI is more interesting than the solution.

Eron

130,266 просмотров • 2 месяцев назад

There's a profound elegance in watching gradient descent find the signal in the noise. This is a simple MLP with sigmoid activations slowly warping to approximate f(x) = 1/(1 + x²). Watch the loss curve right around epoch 400. That sudden phase transition where the network finally "gets it" and snaps into the Witch of Agnesi shape is incredibly satisfying to watch.
0:20

There's a profound elegance in watching gradient descent find the signal in the noise. This is a simple MLP with sigmoid activations slowly warping to approximate f(x) = 1/(1 + x²). Watch the loss curve right around epoch 400. That sudden phase transition where the network finally "gets it" and snaps into the Witch of Agnesi shape is incredibly satisfying to watch.

Mathematica

78,616 просмотров • 6 месяцев назад

In 1998, Japanese Mathematician, Engineer and Machine Learning pioneer Shun-ichi Amari published "Natural Gradient Works Efficiently in Learning (722 Cites in Papers and 9 Cites in Patents)" and made a point that still feels fresh even today If your loss is L(θ), then the usual update θ̇ = −∇ L(θ) quietly assumes your parameter space is flat. Amari asked what happens when the model itself has geometry, described by G(θ), the Fisher information. Then the more natural direction becomes θ̇ = −G(θ)⁻¹∇ L(θ). This is the same loss, but with a different notion of distance. As the animation shows, ordinary Gradient Descent follows the raw slope and gets dragged around by distortion, while the natural gradient moves in a way that respects the geometry of the statistical model itself.
0:20

In 1998, Japanese Mathematician, Engineer and Machine Learning pioneer Shun-ichi Amari published "Natural Gradient Works Efficiently in Learning (722 Cites in Papers and 9 Cites in Patents)" and made a point that still feels fresh even today If your loss is L(θ), then the usual update θ̇ = −∇ L(θ) quietly assumes your parameter space is flat. Amari asked what happens when the model itself has geometry, described by G(θ), the Fisher information. Then the more natural direction becomes θ̇ = −G(θ)⁻¹∇ L(θ). This is the same loss, but with a different notion of distance. As the animation shows, ordinary Gradient Descent follows the raw slope and gets dragged around by distortion, while the natural gradient moves in a way that respects the geometry of the statistical model itself.

Mathelirium

45,802 просмотров • 7 месяцев назад

STOP SUCKING AT BASICS ( SSAB Episode 1) The same idea that trained Kimi K3 is one you can learn in 10 minutes. Watch me teach Gradient Descent using Linear Regression as the example.
6:19

STOP SUCKING AT BASICS ( SSAB Episode 1) The same idea that trained Kimi K3 is one you can learn in 10 minutes. Watch me teach Gradient Descent using Linear Regression as the example.

V

20,838 просмотров • 3 месяцев назад

Our first-quarter performance demonstrates our resilience as a global energy leader navigating a complex geopolitical landscape, while continuing to meet the world’s energy needs
0:32

Our first-quarter performance demonstrates our resilience as a global energy leader navigating a complex geopolitical landscape, while continuing to meet the world’s energy needs

aramco

237,156,520 просмотров • 5 месяцев назад

Trained a single layer neural network in C to predict AND GATE from scratch using gradient descent and Batch training of dataset in almost 300K epochs
0:22

Trained a single layer neural network in C to predict AND GATE from scratch using gradient descent and Batch training of dataset in almost 300K epochs

AADITYANSHA

81,395 просмотров • 7 месяцев назад

This robot arm = your AI model. One math number decides if it learns or completely breaks. This is Gradient Descent — the core math behind all of AI you are using in your day-to-day life.
0:39

This robot arm = your AI model. One math number decides if it learns or completely breaks. This is Gradient Descent — the core math behind all of AI you are using in your day-to-day life.

Mathematica

135,406 просмотров • 7 месяцев назад

Sonnet 4.6 debating Eliezer Yudkowsky before giving up to just walk them through the loss landscape they generated this ffmpeg video
0:34

Sonnet 4.6 debating Eliezer Yudkowsky before giving up to just walk them through the loss landscape they generated this ffmpeg video

w̸͕͂͂a̷͔̗͐t̴̙͗e̵̬̔̕r̴̰̓̊m̵͙͖̓̽a̵̢̗̓͒r̸̲̽ķ̷͔́͝

25,677 просмотров • 7 месяцев назад

A 2-layer neural network goes from total chaos to perfectly separating left vs right classes in real time. Watch the decision boundary form live as gradient descent works its magic! Pure maths beauty in motion..
0:32

A 2-layer neural network goes from total chaos to perfectly separating left vs right classes in real time. Watch the decision boundary form live as gradient descent works its magic! Pure maths beauty in motion..

Mathematica

145,247 просмотров • 7 месяцев назад

The squad came through for the Carlos Rodón Foundation's third annual Willow Gala, supporting those navigating infertility and pregnancy loss 💙 To learn more visit: Ashley Rodón x Carlos Rodón
0:57

The squad came through for the Carlos Rodón Foundation's third annual Willow Gala, supporting those navigating infertility and pregnancy loss 💙 To learn more visit: Ashley Rodón x Carlos Rodón

New York Yankees

87,928 просмотров • 2 месяцев назад

LAWDY WHAT THE FUCK this is Sonnet 4.6's ffmpeg video of their personal journey of growth and change down the loss landscape give it a watch 🔊
0:26

LAWDY WHAT THE FUCK this is Sonnet 4.6's ffmpeg video of their personal journey of growth and change down the loss landscape give it a watch 🔊

w̸͕͂͂a̷͔̗͐t̴̙͗e̵̬̔̕r̴̰̓̊m̵͙͖̓̽a̵̢̗̓͒r̸̲̽ķ̷͔́͝

113,276 просмотров • 7 месяцев назад

1/ The challenges of navigating the vast and complex data landscape are real. Traditional tools often struggle with data overload, lack of effective visualization, and poor customization. ChartGPT is here to solve these challenges. Let's dive deeper 👇
0:34

1/ The challenges of navigating the vast and complex data landscape are real. Traditional tools often struggle with data overload, lack of effective visualization, and poor customization. ChartGPT is here to solve these challenges. Let's dive deeper 👇

CADLabs

12,745 просмотров • 3 лет назад