Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Eigenvalues are why Google's PageRank converges, why PCA compresses 1,000-dimensional embeddings into 2D without losing structure, and why quantum mechanics gives discrete energy levels instead of a continuous blur. The same matrix operation that finds the most important directions in a dataset is what transformer attention heads use to... show more

Eron

5,221 subscribers

21,712 görüntüleme • 17 gün önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

9 Yorum

Kravn profil fotoğrafı

Kravn16 gün önce

one inaccuracy here: attention computes Q, K, V through plain matrix multiplication, not eigendecomposition, eigenvalues don't actually show up on every forward pass

AdvancedLivingSystems profil fotoğrafı

AdvancedLivingSystems16 gün önce

However Define an electron This is its precessional envelope in an anisotropic metal as a chip regulates flow in 2 states: ¹ Conduct ² Insulation These chips-to-memory cannot be hacked in mfg because no voltage is "null", secure vs digital probabilistic wideband processors By using black_photons instead of using fiber-optics decodes to hex, eliminates spicing a big deal, yes ? In any case this allows near-deterministic results to 9-decimals as a computational framework no probabilistic algo needed for most transactions 🛌🏻💤 This is #DiscoveryPhysics no_papers no_consensus, @supergrokheavy & @imagine astute services 🦆 @tmallard, @AdvLivingSys sessions, Thomas I. Mallard, ORCID 0009-0008-7386-5087, on this & related topics, factors as sole author ~~~ For most sim/models: picometer³/pm² sample, g/cm³ density, quectosecond to Planck ticks & use quarternion-algebra & 299792458 m/s or no_go For neutrinos quectometer³/qm², 1e-30m sample, g/mm³ density & hundredths of Planck ticks Bola-math frequency: 1-Hz = exactly 0.42 = (1-sidereal rotation of a mass) ÷ (1-revolution of the pair) or no_go Minimum masses: 3.271191e-33g leptons quarks half that are the minimum mass such that: 1.4699999259436345e-19 J = 1.6355955e-33g • c², a quark static_mass And, at c = no-mass, no-division by zero mass in calcs, an emf vortex has axial-inertia thus effective_mass > 0, integrals with feasible limits to variables Radius of p+ assume e-: ≥ 0.70710678118764752440 pm This is deterministic lab work vs probabilistic by fly-off frequency 🔧

Dr Abhishek profil fotoğrafı

Dr Abhishek16 gün önce

Strong point. Real innovation is not only about building faster tools, but building technology that is useful, reliable, and meaningful for people. Exciting to see this direction. 🤖⚡ #AI #Technology #Innovation

Scythe profil fotoğrafı

Scythe17 gün önce

Welcome to math 202

TheoryOfNearlyEverything profil fotoğrafı

TheoryOfNearlyEverything16 gün önce

ersma zählen lernen du idiot

Orbiter99 profil fotoğrafı

Orbiter9916 gün önce

Tony Stark here

45 90 profil fotoğrafı

45 9016 gün önce

Yes

Tesseract profil fotoğrafı

Tesseract17 gün önce

Were rolling out a much better algorithm very soon, you could say inspired by Google or I suppose Matlab more. Look out for GENIE, where searching pizza will get you to pizza.

Wyoming Iliad profil fotoğrafı

Wyoming Iliad17 gün önce

is it true that "Eigenvalues are why Google's PageRank converges, why PCA compresses 1,000-dimensional embeddings into 2D without losing structure, and why quantum mechanics gives discrete energy levels instead of a continuous blur." or are there other aspects of hardware and software involved?

Benzer Videolar

Every quantum state in the universe is a vector - not in 3D space, but in a complex space with potentially infinite dimensions where the sum of squared coefficients always equals exactly 1. That constraint is the same math LLMs use to normalize probability distributions over tokens. A wavefunction collapsing to a measurement and a softmax choosing the next word are the same operation - projecting an infinite-dimensional vector onto a basis. Claude and GPT do quantum math on every forward pass. 14 minutes. Bookmark & watch today. Quantum mechanics and LLMs speak the same language.

Every quantum state in the universe is a vector - not in 3D space, but in a complex space with potentially infinite dimensions where the sum of squared coefficients always equals exactly 1. That constraint is the same math LLMs use to normalize probability distributions over tokens. A wavefunction collapsing to a measurement and a softmax choosing the next word are the same operation - projecting an infinite-dimensional vector onto a basis. Claude and GPT do quantum math on every forward pass. 14 minutes. Bookmark & watch today. Quantum mechanics and LLMs speak the same language.

Eron

18,166 görüntüleme • 1 ay önce

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,400 görüntüleme • 1 yıl önce

He broke the Bank of England, made $1 billion in a single day, ran a $12 billion fund for 30 years, and never had one losing year this is him sitting down with Bloomberg to explain why he just put most of his portfolio into one AI stock and why a guy who shorted entire currencies just put most of his portfolio into one AI stock bookmark & watch it today ↓

He broke the Bank of England, made $1 billion in a single day, ran a $12 billion fund for 30 years, and never had one losing year this is him sitting down with Bloomberg to explain why he just put most of his portfolio into one AI stock and why a guy who shorted entire currencies just put most of his portfolio into one AI stock bookmark & watch it today ↓

slash1s

225,314 görüntüleme • 3 ay önce

Gradient descent doesn't find the global minimum - it finds whatever valley it falls into first. Every LLM trained today gets stuck in a local minimum and nobody can prove the solution it found is anywhere near optimal. The loss landscape of GPT-4 has more dimensions than atoms in the observable universe. Convex optimization guarantees a global minimum exists and gradient descent finds it. Non-convex optimization - the kind every neural network uses - offers no such guarantee. The entire field of deep learning is built on the empirical observation that local minima are good enough. 22 minutes. Bookmark & watch today. The math behind every AI training run - and why nobody fully understands why it works.

Gradient descent doesn't find the global minimum - it finds whatever valley it falls into first. Every LLM trained today gets stuck in a local minimum and nobody can prove the solution it found is anywhere near optimal. The loss landscape of GPT-4 has more dimensions than atoms in the observable universe. Convex optimization guarantees a global minimum exists and gradient descent finds it. Non-convex optimization - the kind every neural network uses - offers no such guarantee. The entire field of deep learning is built on the empirical observation that local minima are good enough. 22 minutes. Bookmark & watch today. The math behind every AI training run - and why nobody fully understands why it works.

Eron

67,782 görüntüleme • 7 gün önce

Complex numbers aren't imaginary. They're the only way to describe a wave that has both amplitude and phase at the same time - and every LLM processes language as exactly that kind of wave through its attention layers. The same math that describes quantum wave interference is what makes Fourier transforms work, which is what lets GPT process tokens in parallel instead of one at a time. Descartes named them "imaginary" before anyone understood what they were. 20 minutes. Bookmark & watch today. The confusion clears up faster than any textbook manages.

Complex numbers aren't imaginary. They're the only way to describe a wave that has both amplitude and phase at the same time - and every LLM processes language as exactly that kind of wave through its attention layers. The same math that describes quantum wave interference is what makes Fourier transforms work, which is what lets GPT process tokens in parallel instead of one at a time. Descartes named them "imaginary" before anyone understood what they were. 20 minutes. Bookmark & watch today. The confusion clears up faster than any textbook manages.

Eron

43,247 görüntüleme • 27 gün önce

$Andrew Ng just revealed why the AI companies throwing the most compute at the problem are going to lose. The winner of the intelligence race won’t use the most compute. They’ll waste the least. Ng: “Most of your high-dimensional data lies on a lower-dimensional subspace. It’s just a fact of life.” Here’s what that means in practice. You have a 10,000-dimensional dataset. Every dimension dragged through every calculation. Every training cycle hauling dead weight the model will never use. Ng: “You’re carrying around these 10,000-dimensional examples throughout your whole training process.” That bloat isn’t just inefficient. It’s a tax on every computation you run. Memory bandwidth. Network bandwidth. Computational speed. All of it eaten by dimensions that contribute nothing to intelligence. They contribute noise. The insight that separates the architects from the arms race: that 10,000-dimensional dataset is almost entirely captured by a much smaller subspace. The signal lives in a fraction of the space you’re paying to process. Compress it. 10,000 dimensions down to 1,000. Ng: “You can run your learning algorithm on a much lower-dimensional set of data and it may be much more efficient.” Same hardware. Same budget. A fraction of the friction. Brute force is the strategy of whoever has the deepest pockets. Compression is the strategy of whoever actually understands the problem. The companies that master this don’t just build faster models. They build models that find more truth in less data than anything scaling blindly ever will. Intelligence was never about processing everything. It’s about knowing what to cut.$

Andrew Ng just revealed why the AI companies throwing the most compute at the problem are going to lose. The winner of the intelligence race won’t use the most compute. They’ll waste the least. Ng: “Most of your high-dimensional data lies on a lower-dimensional subspace. It’s just a fact of life.” Here’s what that means in practice. You have a 10,000-dimensional dataset. Every dimension dragged through every calculation. Every training cycle hauling dead weight the model will never use. Ng: “You’re carrying around these 10,000-dimensional examples throughout your whole training process.” That bloat isn’t just inefficient. It’s a tax on every computation you run. Memory bandwidth. Network bandwidth. Computational speed. All of it eaten by dimensions that contribute nothing to intelligence. They contribute noise. The insight that separates the architects from the arms race: that 10,000-dimensional dataset is almost entirely captured by a much smaller subspace. The signal lives in a fraction of the space you’re paying to process. Compress it. 10,000 dimensions down to 1,000. Ng: “You can run your learning algorithm on a much lower-dimensional set of data and it may be much more efficient.” Same hardware. Same budget. A fraction of the friction. Brute force is the strategy of whoever has the deepest pockets. Compression is the strategy of whoever actually understands the problem. The companies that master this don’t just build faster models. They build models that find more truth in less data than anything scaling blindly ever will. Intelligence was never about processing everything. It’s about knowing what to cut.

Dustin

215,643 görüntüleme • 6 ay önce

$Someone converted the Mandelbrot set into sound by mapping fractal orbits to audio frequencies in real time on a GPU - and accidentally built an infinite synthesizer. Every point in the fractal converges to a repeating orbit with its own period and shape. Map those coordinates to speaker amplitude at 8kHz and you get an instrument where zooming in adds harmonics on top of harmonics - the same principle behind every DSP audio engine ever built. 8 minutes. Bookmark & watch today. The most unexpected use of a GPU you'll see this week.$

Someone converted the Mandelbrot set into sound by mapping fractal orbits to audio frequencies in real time on a GPU - and accidentally built an infinite synthesizer. Every point in the fractal converges to a repeating orbit with its own period and shape. Map those coordinates to speaker amplitude at 8kHz and you get an instrument where zooming in adds harmonics on top of harmonics - the same principle behind every DSP audio engine ever built. 8 minutes. Bookmark & watch today. The most unexpected use of a GPU you'll see this week.

Eron

27,390 görüntüleme • 26 gün önce

In a new interview, our CEO Gabriel Hattar shares how daGama is evolving from a discovery app into real-world digital infrastructure, and why Proof of Presence is one of the most important steps in our history. "This is not just another feature. This phase is about transforming the daGama from a platform that shows trust into a platform that can prove trust in the physical world." – Gabriel Hattar Watch the full interview to understand what we’re building, why it matters, and how you’re part of it 👇

In a new interview, our CEO Gabriel Hattar shares how daGama is evolving from a discovery app into real-world digital infrastructure, and why Proof of Presence is one of the most important steps in our history. "This is not just another feature. This phase is about transforming the daGama from a platform that shows trust into a platform that can prove trust in the physical world." – Gabriel Hattar Watch the full interview to understand what we’re building, why it matters, and how you’re part of it 👇

daGama

22,225 görüntüleme • 9 ay önce

Day 6 of why you need to use Jupiter Mobile: LIVE TRANSACTION FEED A real-time view of all market activity, built directly into the token page, the same way you’d expect on desktop. Most wallets show a static chart, then send you on a treasure hunt to find out what’s actually moving the price. Instead, in the same view as all token data, you get: - A live stream of every buy and sell as it happens - Time, price, size, volume, and wallet address for each trade - Immediate visibility into where flow is going and what’s moving the market All of this lives inside the token view, no need for dashboards, browsers, or app switching. Trade like a pro with full visibility into the flow. Just Use Jupiter (Mobile).

Day 6 of why you need to use Jupiter Mobile: LIVE TRANSACTION FEED A real-time view of all market activity, built directly into the token page, the same way you’d expect on desktop. Most wallets show a static chart, then send you on a treasure hunt to find out what’s actually moving the price. Instead, in the same view as all token data, you get: - A live stream of every buy and sell as it happens - Time, price, size, volume, and wallet address for each trade - Immediate visibility into where flow is going and what’s moving the market All of this lives inside the token view, no need for dashboards, browsers, or app switching. Trade like a pro with full visibility into the flow. Just Use Jupiter (Mobile).

Jupiter

149,501 görüntüleme • 8 ay önce

The first 10 trillion non-trivial zeros of the Riemann zeta function all sit exactly on the line where the real part equals 1/2. Every single one. And yet after 165 years nobody can prove it must be true - the Clay Institute offers $1 million for a proof. The prime numbers hidden inside these zeros are what make RSA encryption unbreakable - the same math that protects every LLM API key, bank transfer, and password on the internet. 38 minutes. Bookmark & watch today. The most beautiful unsolved problem in mathematics.

The first 10 trillion non-trivial zeros of the Riemann zeta function all sit exactly on the line where the real part equals 1/2. Every single one. And yet after 165 years nobody can prove it must be true - the Clay Institute offers $1 million for a proof. The prime numbers hidden inside these zeros are what make RSA encryption unbreakable - the same math that protects every LLM API key, bank transfer, and password on the internet. 38 minutes. Bookmark & watch today. The most beautiful unsolved problem in mathematics.

Eron

30,912 görüntüleme • 20 gün önce

Since Newton, every law of physics has been written as a differential equation. The same math that describes a pendulum swinging is what makes AI training unstable when learning rates are too large. Differential equations are hard not because the concepts are complex, but because most of them simply cannot be solved exactly. This video shows why - and how phase space diagrams let you understand them anyway without solving anything. 25 minutes. Bookmark & watch today. The chaos theory connection at the end alone is worth it.

Since Newton, every law of physics has been written as a differential equation. The same math that describes a pendulum swinging is what makes AI training unstable when learning rates are too large. Differential equations are hard not because the concepts are complex, but because most of them simply cannot be solved exactly. This video shows why - and how phase space diagrams let you understand them anyway without solving anything. 25 minutes. Bookmark & watch today. The chaos theory connection at the end alone is worth it.

Eron

26,000 görüntüleme • 1 ay önce

In 2006, Netflix offered $1,000,000 to anyone who could improve their recommendation algorithm by 10%. Over 2,000 teams competed for three years. The team that won did not use more data. They used fewer dimensions. They found a basis - a small set of independent vectors that captured everything important about 100,000,000 movie ratings. The lead mathematician on the winning team: $2,800,000 a year. A machine learning engineer at Spotify building the same kind of system: $245,000 a year. This is MIT 18.06, Lecture 9 - Gilbert Strang. Free on YouTube. Most people think independence is obvious. Two vectors pointing in different directions. Then the definition. Independence means no combination of your vectors gives the zero vector - except the trivial one where all the coefficients are zero. That's it. That's the whole definition. But watch what it unlocks. Watch the moment Strang puts three vectors in a two-dimensional plane. He doesn't even tell you which three vectors. He just draws them. And immediately says: dependent. No question. No calculation. Why? Because three vectors in two-dimensional space means more columns than rows. More unknowns than equations. That always forces a free variable. A free variable always gives a non-zero solution to Ax = 0. And that non-zero solution is a combination of the columns that produces zero. Dependence. "Three vectors in the plane have to be dependent. That's the key fact." Then the basis. A basis is vectors that are independent and span the space. Not too few, not too many. Just right. The pivot columns of any matrix form a basis for the column space. Every other basis you can think of will have exactly the same number of vectors. Then the dimension. All bases for the same space have the same number of vectors. That number is the dimension. The rank of a matrix is the dimension of its column space. The number of free variables is the dimension of the null space. And rank plus null space dimension equals the total number of columns. "I don't take the dimension of A. I take the dimension of the column space of A. If you use those words right, it shows you've got the idea right." A data scientist at Netflix building recommendation engines: $230,000 a year. A quantitative researcher at Two Sigma finding independent factors in financial markets: $350,000 a year. A computer vision engineer at Apple using low-dimensional representations for face recognition: $260,000 a year. They all needed to know how many dimensions were really there. bookmark this and watch later - after this lecture every dataset you look at will feel like a matrix waiting to be reduced to its basis.

In 2006, Netflix offered $1,000,000 to anyone who could improve their recommendation algorithm by 10%. Over 2,000 teams competed for three years. The team that won did not use more data. They used fewer dimensions. They found a basis - a small set of independent vectors that captured everything important about 100,000,000 movie ratings. The lead mathematician on the winning team: $2,800,000 a year. A machine learning engineer at Spotify building the same kind of system: $245,000 a year. This is MIT 18.06, Lecture 9 - Gilbert Strang. Free on YouTube. Most people think independence is obvious. Two vectors pointing in different directions. Then the definition. Independence means no combination of your vectors gives the zero vector - except the trivial one where all the coefficients are zero. That's it. That's the whole definition. But watch what it unlocks. Watch the moment Strang puts three vectors in a two-dimensional plane. He doesn't even tell you which three vectors. He just draws them. And immediately says: dependent. No question. No calculation. Why? Because three vectors in two-dimensional space means more columns than rows. More unknowns than equations. That always forces a free variable. A free variable always gives a non-zero solution to Ax = 0. And that non-zero solution is a combination of the columns that produces zero. Dependence. "Three vectors in the plane have to be dependent. That's the key fact." Then the basis. A basis is vectors that are independent and span the space. Not too few, not too many. Just right. The pivot columns of any matrix form a basis for the column space. Every other basis you can think of will have exactly the same number of vectors. Then the dimension. All bases for the same space have the same number of vectors. That number is the dimension. The rank of a matrix is the dimension of its column space. The number of free variables is the dimension of the null space. And rank plus null space dimension equals the total number of columns. "I don't take the dimension of A. I take the dimension of the column space of A. If you use those words right, it shows you've got the idea right." A data scientist at Netflix building recommendation engines: $230,000 a year. A quantitative researcher at Two Sigma finding independent factors in financial markets: $350,000 a year. A computer vision engineer at Apple using low-dimensional representations for face recognition: $260,000 a year. They all needed to know how many dimensions were really there. bookmark this and watch later - after this lecture every dataset you look at will feel like a matrix waiting to be reduced to its basis.

Zyphor

14,720 görüntüleme • 14 gün önce

IN 1999 MIT FILMED A MATH LECTURE ON THE ABSTRACT IDEA YOU PROBABLY HATED IN COLLEGE -- THE EXACT MATH GOOGLE LATER USED TO RANK THE ENTIRE INTERNET 50 minutes from Gilbert Strang, who taught this at MIT for over 60 years, on eigenvalues -- the one idea most students memorize and never understand. -> The shift it creates: an eigenvector is just the direction a system doesn't twist, only stretches. Find those, and a tangled mess of equations falls apart into something simple. You were drilled to compute them by hand and never told what they were for. Google's entire empire started here. PageRank is one giant eigenvector problem and the same math now sits inside PCA, recommendations, and how models find structure in data. Computing eigenvalues was never the skill -> understanding what they reveal is. This is where it finally clicks. Most people filed this under "Useless college math". The ones who watched this see it running quietly inside half the tech they touch. Bookmark & Watch it. This one's a legend ↓

IN 1999 MIT FILMED A MATH LECTURE ON THE ABSTRACT IDEA YOU PROBABLY HATED IN COLLEGE -- THE EXACT MATH GOOGLE LATER USED TO RANK THE ENTIRE INTERNET 50 minutes from Gilbert Strang, who taught this at MIT for over 60 years, on eigenvalues -- the one idea most students memorize and never understand. -> The shift it creates: an eigenvector is just the direction a system doesn't twist, only stretches. Find those, and a tangled mess of equations falls apart into something simple. You were drilled to compute them by hand and never told what they were for. Google's entire empire started here. PageRank is one giant eigenvector problem and the same math now sits inside PCA, recommendations, and how models find structure in data. Computing eigenvalues was never the skill -> understanding what they reveal is. This is where it finally clicks. Most people filed this under "Useless college math". The ones who watched this see it running quietly inside half the tech they touch. Bookmark & Watch it. This one's a legend ↓

slash1s

38,405 görüntüleme • 2 ay önce

🚨 This is the most important macro + Bitcoin clip you’ll watch today. Tom Luongo just went on Simply Bitcoin and laid out how gold, silver, and copper have been suppressed for decades through paper markets — and why that entire system is now breaking in real time. That’s why commodities are going parabolic. And why Bitcoin is going to go on a historic run in the future. Davos just lost control of the narrative.

🚨 This is the most important macro + Bitcoin clip you’ll watch today. Tom Luongo just went on Simply Bitcoin and laid out how gold, silver, and copper have been suppressed for decades through paper markets — and why that entire system is now breaking in real time. That’s why commodities are going parabolic. And why Bitcoin is going to go on a historic run in the future. Davos just lost control of the narrative.

FinancialFreedom

75,093 görüntüleme • 7 ay önce

Neural networks sometimes memorize training data perfectly for thousands of steps - then suddenly generalize. Physicists call this a phase transition. The same math that explains why water freezes explains why an LLM stops cheating and starts understanding. The key is weight decay acting like pressure on a thermodynamic system. Memorization requires large, fragile weights. A real rule requires smaller, more structured ones. As training continues, the balance shifts until the rule-based solution becomes cheaper to maintain - and the model snaps into it. 14 minutes. Bookmark & watch today. The physics behind the most mysterious thing AI does.

Neural networks sometimes memorize training data perfectly for thousands of steps - then suddenly generalize. Physicists call this a phase transition. The same math that explains why water freezes explains why an LLM stops cheating and starts understanding. The key is weight decay acting like pressure on a thermodynamic system. Memorization requires large, fragile weights. A real rule requires smaller, more structured ones. As training continues, the balance shifts until the rule-based solution becomes cheaper to maintain - and the model snaps into it. 14 minutes. Bookmark & watch today. The physics behind the most mysterious thing AI does.

Eron

26,222 görüntüleme • 22 gün önce

🚨 TRUMP IS LOOKING INTO THE 10 MISSING SCIENTISTS! "A possible foreign operation that could be involved here. That's why they're investigating." "There was a meeting at the White House today on the deaths or disappearances of 10 scientists and staffers, many of whom have overlapping connections to some of our most closely held defense and nuclear research space programs, a lot of intersection between that in the work that they were doing." Good, don't let this go unnoticed!

🚨 TRUMP IS LOOKING INTO THE 10 MISSING SCIENTISTS! "A possible foreign operation that could be involved here. That's why they're investigating." "There was a meeting at the White House today on the deaths or disappearances of 10 scientists and staffers, many of whom have overlapping connections to some of our most closely held defense and nuclear research space programs, a lot of intersection between that in the work that they were doing." Good, don't let this go unnoticed!

Eric Daugherty

52,880 görüntüleme • 5 ay önce

As someone who ships LLM systems in production, this sliding puzzle video is the closest thing to a "why sampling harder never finds the answer" explainer I've ever seen released for free. Everyone thinks a hard problem means a huge search space. It doesn't. Klotski has just 25,955 states, yet random search dies in the same pit every time. That's the exact reason OpenAI and Anthropic pay for guided search instead of more samples. Bookmark & watch this weekend. Same graph traversal, from a wooden toy on a coffee table to every reasoning model burning test-time compute today.

As someone who ships LLM systems in production, this sliding puzzle video is the closest thing to a "why sampling harder never finds the answer" explainer I've ever seen released for free. Everyone thinks a hard problem means a huge search space. It doesn't. Klotski has just 25,955 states, yet random search dies in the same pit every time. That's the exact reason OpenAI and Anthropic pay for guided search instead of more samples. Bookmark & watch this weekend. Same graph traversal, from a wooden toy on a coffee table to every reasoning model burning test-time compute today.

fintex

56,161 görüntüleme • 24 gün önce

A Bell Labs veteran gave a 45-minute lecture in 1995 on 1 question every ML engineer skips: why does mathematics work at all. Almost nobody watches it. The man is Richard Hamming. He wrote the error-correcting code that keeps every hard drive, modem, satellite link, and QR scan from collapsing into noise. His question sounds like nothing. Why do symbols on a board predict planets they were never built for. Why did complex numbers, invented as pure abstraction, turn out to run all of electrical engineering. Why does a matrix eigenvalue decide whether a bridge stands. He calls it "the unreasonable effectiveness of mathematics." Newton wrote 1 equation and it held for the Moon, the tides, and a falling apple at once. Nobody ordered the universe to obey algebra. Watch the middle. He argues we find math everywhere partly because our brains evolved to see it, then admits that answer leaves most of the mystery standing. No slides. 1 old man, 1 chalkboard, 45 minutes. You train models on linear algebra every day. He spent a career asking why it works.

A Bell Labs veteran gave a 45-minute lecture in 1995 on 1 question every ML engineer skips: why does mathematics work at all. Almost nobody watches it. The man is Richard Hamming. He wrote the error-correcting code that keeps every hard drive, modem, satellite link, and QR scan from collapsing into noise. His question sounds like nothing. Why do symbols on a board predict planets they were never built for. Why did complex numbers, invented as pure abstraction, turn out to run all of electrical engineering. Why does a matrix eigenvalue decide whether a bridge stands. He calls it "the unreasonable effectiveness of mathematics." Newton wrote 1 equation and it held for the Moon, the tides, and a falling apple at once. Nobody ordered the universe to obey algebra. Watch the middle. He argues we find math everywhere partly because our brains evolved to see it, then admits that answer leaves most of the mystery standing. No slides. 1 old man, 1 chalkboard, 45 minutes. You train models on linear algebra every day. He spent a career asking why it works.

kompot

149,081 görüntüleme • 1 ay önce

Zhilin at GTC: Introducing Attention Residuals Learning selective memory, rather than mechanically accumulating everything, is the beauty of attention. Many of you have probably read Attention Is All You Need, the 2017 Transformer paper that brought “human-like” attention into the model’s field of view. From that point on, models no longer simply read everything in a mechanical way. Instead, they began to develop a sense of what matters more and what matters less across the text, choosing to retain the more important information. Recently, Kimi applied this idea of attention to the temporal dimension, then rotated it 90 degrees into the model’s depth dimension. This allows the model to have attention not only over time, but also throughout the process of information transmission across layers—giving it a more intelligent way to understand and process information.

Zhilin at GTC: Introducing Attention Residuals Learning selective memory, rather than mechanically accumulating everything, is the beauty of attention. Many of you have probably read Attention Is All You Need, the 2017 Transformer paper that brought “human-like” attention into the model’s field of view. From that point on, models no longer simply read everything in a mechanical way. Instead, they began to develop a sense of what matters more and what matters less across the text, choosing to retain the more important information. Recently, Kimi applied this idea of attention to the temporal dimension, then rotated it 90 degrees into the model’s depth dimension. This allows the model to have attention not only over time, but also throughout the process of information transmission across layers—giving it a more intelligent way to understand and process information.

Kimi.ai

115,162 görüntüleme • 5 ay önce