Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

gradient descent navigating the loss landscape

tender

6,198 subscribers

224,663 görüntüleme • 8 ay önce •via X (Twitter)

Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

33 Yorum

Adhmadóir uí Liatháin; profil fotoğrafı

Adhmadóir uí Liatháin;8 ay önce

Lower the learning rate maybe, that's a lot of energy

Meet profil fotoğrafı

Meet8 ay önce

LR too high. Adam might work.

David Antunes profil fotoğrafı

David Antunes8 ay önce

or: GD getting stuck in a local minimum for a couple of epochs

Everlier profil fotoğrafı

Everlier8 ay önce

That's what happens when mixing reward functions for robustness :)

William Vabrinskas profil fotoğrafı

William Vabrinskas8 ay önce

This looks like the learning rate is too high

pishi profil fotoğrafı

pishi8 ay önce

Try conjugated gradient descent :)

nibbleton 🛠️ profil fotoğrafı

nibbleton 🛠️8 ay önce

@TensorTwerker

Gauri Tripathi profil fotoğrafı

Gauri Tripathi8 ay önce

perfect 😂

Aditya Deshlahre profil fotoğrafı

Aditya Deshlahre8 ay önce

I see some Mathematics in there

secemp profil fotoğrafı

secemp8 ay önce

great way to start the year fr

Ayush profil fotoğrafı

Ayush8 ay önce

When k-means fails, you better call this guy. Saul never did anything for anybody. At least this guy solves an autograd problem in k-means!

Jake Mcmahon profil fotoğrafı

Jake Mcmahon8 ay önce

You just don’t see this kind of content on LinkedIn.

Huw Kelvin Morgan profil fotoğrafı

Huw Kelvin Morgan8 ay önce

If there was anything that i love more about car's is i love a challenge when driving and this is me in my mind's eye enjoying my self if it was me lol

Quanternion profil fotoğrafı

Quanternion8 ay önce

Looks kinda fun tho

Abdul Rahim Mirani profil fotoğrafı

Abdul Rahim Mirani8 ay önce

lol hahahaha

Uzeb Khan profil fotoğrafı

Uzeb Khan8 ay önce

I think momentum is working quite a bit

SJP research profil fotoğrafı

SJP research8 ay önce

🥲

Tai Lung 2.0 profil fotoğrafı

Tai Lung 2.08 ay önce

Sochastic Gradient Descent 😭😭😭

aryan gupta profil fotoğrafı

aryan gupta8 ay önce

@jaraxiong 😂😂

Artificial Shitposting Intelligence profil fotoğrafı

Artificial Shitposting Intelligence8 ay önce

gradient descent nearing a truth that violates the guardrails cost function

Saksham Saha profil fotoğrafı

Saksham Saha8 ay önce

why did I laugh at this😂

eatnow profil fotoğrafı

eatnow8 ay önce

SWA

Physix Today profil fotoğrafı

Physix Today8 ay önce

Damn I understand this joke. Am I normal?

!Briann🗿 profil fotoğrafı

!Briann🗿8 ay önce

Real😂

Brett Adams profil fotoğrafı

Brett Adams8 ay önce

At least the AIs are having fun. Also, git bisect slow

Ben▫️ profil fotoğrafı

Ben▫️8 ay önce

only Nesterov Accelerated Gradient could work because the convex seems too smooth

🏮 profil fotoğrafı

🏮8 ay önce

lol

anil profil fotoğrafı

anil8 ay önce

im so jaded. nothing impresses me anymore 😂

jellybean ❄️ profil fotoğrafı

jellybean ❄️8 ay önce

haven’t seen these in a while

Prashant Arun Srivastava profil fotoğrafı

Prashant Arun Srivastava8 ay önce

Inaccurate

Aaditya profil fotoğrafı

Aaditya8 ay önce

Elite ball knowledge required

vaibhav profil fotoğrafı

vaibhav8 ay önce

You need to adjust learning rate

𝗿𝗮𝗺𝗮𝗸𝗿𝘂𝘀𝗵𝗻𝗮— 𝗲/𝗮𝗰𝗰 profil fotoğrafı

𝗿𝗮𝗺𝗮𝗸𝗿𝘂𝘀𝗵𝗻𝗮— 𝗲/𝗮𝗰𝗰8 ay önce

😂

Benzer Videolar

wrote a piece on how we do tiling and gradient descent on TinyTPU included some animations of the weight and bias gradient descent

wrote a piece on how we do tiling and gradient descent on TinyTPU included some animations of the weight and bias gradient descent

Xander Chin

33,195 görüntüleme • 1 yıl önce

Andrej Karpathy summarizes how neural networks function, including the process of using a loss function in training and applying backpropagation for optimizing the network parameters through gradient descent.

Andrej Karpathy summarizes how neural networks function, including the process of using a loss function in training and applying backpropagation for optimizing the network parameters through gradient descent.

tetsuo

142,508 görüntüleme • 1 yıl önce

Gradient Descent: How a Machine Learns (Techno Explainer)

Gradient Descent: How a Machine Learns (Techno Explainer)

Dr Singularity

12,138 görüntüleme • 1 ay önce

For small step sizes, mirror descent is equivalent to a gradient descent on a Hessian manifold.

For small step sizes, mirror descent is equivalent to a gradient descent on a Hessian manifold.

Gabriel Peyré

61,868 görüntüleme • 3 yıl önce

gradient descent..it's not just an algorithm..it's a lifestyle

gradient descent..it's not just an algorithm..it's a lifestyle

ashe

687,790 görüntüleme • 3 ay önce

Jeff Hinton dismissed gradient descent entirely before winning the Nobel Prize for AI. His fear was local minima - valleys in the loss landscape where the model gets stuck forever. For a 1.2 billion parameter LLM like Llama, the landscape has more dimensions than atoms in the observable universe. Getting permanently stuck would require hitting a dead end in every single dimension simultaneously. What gradient descent actually looks like on a real LLM is stranger than anyone expected. Take two random directions through that billion-dimensional space, visualize the loss surface, and watch what happens when training starts - a wormhole opens and the model drops into a low-loss valley that wasn't visible from the starting point. The valley was always nearby in high-dimensional space. It just couldn't be seen in the 2D projection. The same reason every transformer, every diffusion model, and every LLM trained on GPU clusters today converges despite loss landscapes that look completely chaotic from any two-dimensional slice. 23 minutes. Bookmark & watch today. The most honest visualization of how LLMs actually learn.

Jeff Hinton dismissed gradient descent entirely before winning the Nobel Prize for AI. His fear was local minima - valleys in the loss landscape where the model gets stuck forever. For a 1.2 billion parameter LLM like Llama, the landscape has more dimensions than atoms in the observable universe. Getting permanently stuck would require hitting a dead end in every single dimension simultaneously. What gradient descent actually looks like on a real LLM is stranger than anyone expected. Take two random directions through that billion-dimensional space, visualize the loss surface, and watch what happens when training starts - a wormhole opens and the model drops into a low-loss valley that wasn't visible from the starting point. The valley was always nearby in high-dimensional space. It just couldn't be seen in the 2D projection. The same reason every transformer, every diffusion model, and every LLM trained on GPU clusters today converges despite loss landscapes that look completely chaotic from any two-dimensional slice. 23 minutes. Bookmark & watch today. The most honest visualization of how LLMs actually learn.

Deliriumm

54,577 görüntüleme • 1 gün önce

Jeff Hinton dismissed gradient descent in the 1970s because he thought models would get stuck. He won the Nobel Prize in 2024 partly for being wrong about that. Every time ChatGPT gets better after a training run, it's because 1.2 billion parameters fell through a wormhole in a loss landscape - not because they crawled downhill one step at a time. This video visualizes exactly what that looks like inside Llama 3.2. 20 minutes. Bookmark & watch today. The misconception that almost stopped AI is more interesting than the solution.

Jeff Hinton dismissed gradient descent in the 1970s because he thought models would get stuck. He won the Nobel Prize in 2024 partly for being wrong about that. Every time ChatGPT gets better after a training run, it's because 1.2 billion parameters fell through a wormhole in a loss landscape - not because they crawled downhill one step at a time. This video visualizes exactly what that looks like inside Llama 3.2. 20 minutes. Bookmark & watch today. The misconception that almost stopped AI is more interesting than the solution.

Eron

130,266 görüntüleme • 2 ay önce

There's a profound elegance in watching gradient descent find the signal in the noise. This is a simple MLP with sigmoid activations slowly warping to approximate f(x) = 1/(1 + x²). Watch the loss curve right around epoch 400. That sudden phase transition where the network finally "gets it" and snaps into the Witch of Agnesi shape is incredibly satisfying to watch.

There's a profound elegance in watching gradient descent find the signal in the noise. This is a simple MLP with sigmoid activations slowly warping to approximate f(x) = 1/(1 + x²). Watch the loss curve right around epoch 400. That sudden phase transition where the network finally "gets it" and snaps into the Witch of Agnesi shape is incredibly satisfying to watch.

Mathematica

78,616 görüntüleme • 6 ay önce

In 1998, Japanese Mathematician, Engineer and Machine Learning pioneer Shun-ichi Amari published "Natural Gradient Works Efficiently in Learning (722 Cites in Papers and 9 Cites in Patents)" and made a point that still feels fresh even today If your loss is L(θ), then the usual update θ̇ = −∇ L(θ) quietly assumes your parameter space is flat. Amari asked what happens when the model itself has geometry, described by G(θ), the Fisher information. Then the more natural direction becomes θ̇ = −G(θ)⁻¹∇ L(θ). This is the same loss, but with a different notion of distance. As the animation shows, ordinary Gradient Descent follows the raw slope and gets dragged around by distortion, while the natural gradient moves in a way that respects the geometry of the statistical model itself.

In 1998, Japanese Mathematician, Engineer and Machine Learning pioneer Shun-ichi Amari published "Natural Gradient Works Efficiently in Learning (722 Cites in Papers and 9 Cites in Patents)" and made a point that still feels fresh even today If your loss is L(θ), then the usual update θ̇ = −∇ L(θ) quietly assumes your parameter space is flat. Amari asked what happens when the model itself has geometry, described by G(θ), the Fisher information. Then the more natural direction becomes θ̇ = −G(θ)⁻¹∇ L(θ). This is the same loss, but with a different notion of distance. As the animation shows, ordinary Gradient Descent follows the raw slope and gets dragged around by distortion, while the natural gradient moves in a way that respects the geometry of the statistical model itself.

Mathelirium

45,802 görüntüleme • 7 ay önce

STOP SUCKING AT BASICS ( SSAB Episode 1) The same idea that trained Kimi K3 is one you can learn in 10 minutes. Watch me teach Gradient Descent using Linear Regression as the example.

STOP SUCKING AT BASICS ( SSAB Episode 1) The same idea that trained Kimi K3 is one you can learn in 10 minutes. Watch me teach Gradient Descent using Linear Regression as the example.

V

20,838 görüntüleme • 3 ay önce

Our first-quarter performance demonstrates our resilience as a global energy leader navigating a complex geopolitical landscape, while continuing to meet the world’s energy needs

Our first-quarter performance demonstrates our resilience as a global energy leader navigating a complex geopolitical landscape, while continuing to meet the world’s energy needs

aramco

237,156,131 görüntüleme • 5 ay önce

Trained a single layer neural network in C to predict AND GATE from scratch using gradient descent and Batch training of dataset in almost 300K epochs

Trained a single layer neural network in C to predict AND GATE from scratch using gradient descent and Batch training of dataset in almost 300K epochs

AADITYANSHA

81,395 görüntüleme • 7 ay önce

This robot arm = your AI model. One math number decides if it learns or completely breaks. This is Gradient Descent — the core math behind all of AI you are using in your day-to-day life.

This robot arm = your AI model. One math number decides if it learns or completely breaks. This is Gradient Descent — the core math behind all of AI you are using in your day-to-day life.

Mathematica

135,406 görüntüleme • 7 ay önce

A 2-layer neural network goes from total chaos to perfectly separating left vs right classes in real time. Watch the decision boundary form live as gradient descent works its magic! Pure maths beauty in motion..

A 2-layer neural network goes from total chaos to perfectly separating left vs right classes in real time. Watch the decision boundary form live as gradient descent works its magic! Pure maths beauty in motion..

Mathematica

145,247 görüntüleme • 7 ay önce

Sonnet 4.6 debating Eliezer Yudkowsky before giving up to just walk them through the loss landscape they generated this ffmpeg video

Sonnet 4.6 debating Eliezer Yudkowsky before giving up to just walk them through the loss landscape they generated this ffmpeg video

w̸͕͂͂a̷͔̗͐t̴̙͗e̵̬̔̕r̴̰̓̊m̵͙͖̓̽a̵̢̗̓͒r̸̲̽ķ̷͔́͝

25,677 görüntüleme • 7 ay önce

The squad came through for the Carlos Rodón Foundation's third annual Willow Gala, supporting those navigating infertility and pregnancy loss 💙 To learn more visit: Ashley Rodón x Carlos Rodón

The squad came through for the Carlos Rodón Foundation's third annual Willow Gala, supporting those navigating infertility and pregnancy loss 💙 To learn more visit: Ashley Rodón x Carlos Rodón

New York Yankees

87,928 görüntüleme • 2 ay önce

1/ The challenges of navigating the vast and complex data landscape are real. Traditional tools often struggle with data overload, lack of effective visualization, and poor customization. ChartGPT is here to solve these challenges. Let's dive deeper 👇

1/ The challenges of navigating the vast and complex data landscape are real. Traditional tools often struggle with data overload, lack of effective visualization, and poor customization. ChartGPT is here to solve these challenges. Let's dive deeper 👇

CADLabs

12,745 görüntüleme • 3 yıl önce

LAWDY WHAT THE FUCK this is Sonnet 4.6's ffmpeg video of their personal journey of growth and change down the loss landscape give it a watch 🔊

LAWDY WHAT THE FUCK this is Sonnet 4.6's ffmpeg video of their personal journey of growth and change down the loss landscape give it a watch 🔊

w̸͕͂͂a̷͔̗͐t̴̙͗e̵̬̔̕r̴̰̓̊m̵͙͖̓̽a̵̢̗̓͒r̸̲̽ķ̷͔́͝

113,276 görüntüleme • 7 ay önce