Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

3Blue1Brown’s new video explains why every LLM is actually a compression machine. everyone describes pre-training as “next token prediction” but that’s just the surface-level objective. in reality it is a means to making the most efficient text compressor. prediction and compression are two sides of the same coin. when... show more

ℏεsam

85,187 subscribers

120,145 görüntüleme • 3 ay önce •via X (Twitter)

Eğitim Haberler & Politika Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

🚨 ANOTHER MASTERCLASS FROM Grant Sanderson The compressibility of language isn’t just a math curiosity, it’s the hidden engine behind every LLM you use. Grant’s new video reframes Shannon’s entropy through one elegant lens: Prediction IS compression. → The better you predict the next word, the fewer bits you need to store it → Shannon measured English at ~1 bit per character: astonishingly compressible → This is exactly what GPT-style models optimize → Intelligence, in this framing, is compression FUN FACT: Von Neumann told Shannon to name it “entropy” because nobody truly understands it anyway 😄 Decades later, that same concept became the bedrock of modern AI. Deep-dive resources in the 🧵 ↓

🚨 ANOTHER MASTERCLASS FROM Grant Sanderson The compressibility of language isn’t just a math curiosity, it’s the hidden engine behind every LLM you use. Grant’s new video reframes Shannon’s entropy through one elegant lens: Prediction IS compression. → The better you predict the next word, the fewer bits you need to store it → Shannon measured English at ~1 bit per character: astonishingly compressible → This is exactly what GPT-style models optimize → Intelligence, in this framing, is compression FUN FACT: Von Neumann told Shannon to name it “entropy” because nobody truly understands it anyway 😄 Decades later, that same concept became the bedrock of modern AI. Deep-dive resources in the 🧵 ↓

Charly Wargnier

319,799 görüntüleme • 3 ay önce

🚨 Grant Sanderson DID IT AGAIN Language compressibility is not just a neat math trick: it is the core engine of modern LLMs. Grant's latest video boils Shannon's entropy down to a single, powerful idea: Prediction IS compression. → Predict the next word better, use fewer bits to store it → Shannon found English is astonishingly compressible (~1 bit per character) → This is the exact mechanism GPT models run on → Under this framing, intelligence equals compression FUN FACT: Von Neumann told Shannon to use the term "entropy" because no one really understands it. Today, it powers the AI revolution. Deep-dive resources in the 🧵↓

🚨 Grant Sanderson DID IT AGAIN Language compressibility is not just a neat math trick: it is the core engine of modern LLMs. Grant's latest video boils Shannon's entropy down to a single, powerful idea: Prediction IS compression. → Predict the next word better, use fewer bits to store it → Shannon found English is astonishingly compressible (~1 bit per character) → This is the exact mechanism GPT models run on → Under this framing, intelligence equals compression FUN FACT: Von Neumann told Shannon to use the term "entropy" because no one really understands it. Today, it powers the AI revolution. Deep-dive resources in the 🧵↓

Charly Wargnier ♨️

161,480 görüntüleme • 2 ay önce

Ilya Sutskever, co-founder and chief scientist of OpenAI: "The better a neural network can predict the next word, the more it understands it." Everyone calls it autocomplete. Ilya explained why that is a comforting lie. To guess the final word of a murder mystery, the name of the killer, the model has to reason through every clue in the book. Prediction that good is not statistics. It is understanding. Then he said the part that should stop you. The text these models train on is "a projection of the world," and to compress it they learn more and more about people, "the human condition, their hopes, dreams, and motivations." The machine is not memorizing our words. It is quietly building a model of us. That is the whole premise of the fear. The machine will never hate you. It understands you far better than that. The frightening variable was never the intelligence. It was the aim. I wrote the full story: the one fear about AI everyone gets wrong, and the one already here.

Ilya Sutskever, co-founder and chief scientist of OpenAI: "The better a neural network can predict the next word, the more it understands it." Everyone calls it autocomplete. Ilya explained why that is a comforting lie. To guess the final word of a murder mystery, the name of the killer, the model has to reason through every clue in the book. Prediction that good is not statistics. It is understanding. Then he said the part that should stop you. The text these models train on is "a projection of the world," and to compress it they learn more and more about people, "the human condition, their hopes, dreams, and motivations." The machine is not memorizing our words. It is quietly building a model of us. That is the whole premise of the fear. The machine will never hate you. It understands you far better than that. The frightening variable was never the intelligence. It was the aim. I wrote the full story: the one fear about AI everyone gets wrong, and the one already here.

Casper

92,361 görüntüleme • 2 ay önce

🚨| Sir Lewis Hamilton on Mercedes' engine: "What's clear is that they didn't show the engine power through any of the practice, because there's the whole talk of compression ratio, and they've obviously done a really solid job with their engine, which we have as well, but it would be interesting to... I want to understand why it's like two-tenths or more just through power, per sector. And so, if it is a compression thing, I want to understand why the FIA haven't done anything, what's being done to rectify it, but if it's not, and it's just pure [ped], then we have to do a better job." #F1 || #AustralianGP

🚨| Sir Lewis Hamilton on Mercedes' engine: "What's clear is that they didn't show the engine power through any of the practice, because there's the whole talk of compression ratio, and they've obviously done a really solid job with their engine, which we have as well, but it would be interesting to... I want to understand why it's like two-tenths or more just through power, per sector. And so, if it is a compression thing, I want to understand why the FIA haven't done anything, what's being done to rectify it, but if it's not, and it's just pure [ped], then we have to do a better job." #F1 || #AustralianGP

sim

38,390 görüntüleme • 6 ay önce

Martin Casado explains AI as a fundamentally new abstraction layer of computing: "I don't think in the history of computer science that I can recall have we ever abdicated actual reasoning or logic. It's always been a resource. It's been like compute, network, and storage..." "And then the human is putting in the high-level thing, and it's using the compute, network, and storage to calculate the answer." "But now I feel like you're actually abdicating thinking in a way, where you're like, 'Tell me the answer.' Like, 'I'm not even really sure what the question is.'" "This is the first time it feels like a different layer of the stack. Maybe this really is the next abstraction, which is more of a human-level abstraction, which doesn't map directly and so is actually different... You kind of pray to the model god in the right words, and then it produces the answer that just ends up being useful." martin_casado

Martin Casado explains AI as a fundamentally new abstraction layer of computing: "I don't think in the history of computer science that I can recall have we ever abdicated actual reasoning or logic. It's always been a resource. It's been like compute, network, and storage..." "And then the human is putting in the high-level thing, and it's using the compute, network, and storage to calculate the answer." "But now I feel like you're actually abdicating thinking in a way, where you're like, 'Tell me the answer.' Like, 'I'm not even really sure what the question is.'" "This is the first time it feels like a different layer of the stack. Maybe this really is the next abstraction, which is more of a human-level abstraction, which doesn't map directly and so is actually different... You kind of pray to the model god in the right words, and then it produces the answer that just ends up being useful." martin_casado

a16z

36,147 görüntüleme • 26 gün önce

🚨 | Lewis Hamilton: “If it is the compression thing, I wanna understand why the FIA haven’t done anything, what’s been done to rectify it but if it’s not and it’s just pure pace, then we have to do a better job.”

🚨 | Lewis Hamilton: “If it is the compression thing, I wanna understand why the FIA haven’t done anything, what’s been done to rectify it but if it’s not and it’s just pure pace, then we have to do a better job.”

La Gazzetta Ferrari

134,202 görüntüleme • 6 ay önce

🚨 | Lewis Hamilton on Mercedes’ compression ratio: “What’s clear is that they didn’t show the engine power through any of the practice because of the whole talk on the compression issue. They’ve done a really solid job with their engine, which we have as well but I’m trying to understand why it's 2 tenths or more just through power per sector... And so, if it is the compression thing, I wanna understand why the FIA haven't done anything [or] what’s being done to rectify it.”

🚨 | Lewis Hamilton on Mercedes’ compression ratio: “What’s clear is that they didn’t show the engine power through any of the practice because of the whole talk on the compression issue. They’ve done a really solid job with their engine, which we have as well but I’m trying to understand why it's 2 tenths or more just through power per sector... And so, if it is the compression thing, I wanna understand why the FIA haven't done anything [or] what’s being done to rectify it.”

deni

960,486 görüntüleme • 6 ay önce

Mr Shift: What's the true innovation with prediction markets? Kalshi CEO: There's many ways to answer that question. Um there's the Kalshi piece of it, but then there's the prediction market piece of it, which are not exactly the same. “But the... the true innovation in prediction markets, I think, is this notion, like... and, and I think it has been built through a lot of literature, and then I think the prediction markets have come in our generation, like Kalshi and some of the others have proven it” “The best way to answer like all these most pressing questions is like you have to ask the crowd”

Mr Shift: What's the true innovation with prediction markets? Kalshi CEO: There's many ways to answer that question. Um there's the Kalshi piece of it, but then there's the prediction market piece of it, which are not exactly the same. “But the... the true innovation in prediction markets, I think, is this notion, like... and, and I think it has been built through a lot of literature, and then I think the prediction markets have come in our generation, like Kalshi and some of the others have proven it” “The best way to answer like all these most pressing questions is like you have to ask the crowd”

ShaDow | Clipur.com

23,860 görüntüleme • 3 ay önce

World Labs co-founders Justin Johnson and Dr. Fei-Fei Li say LLMs use next-token prediction, but spatial intelligence has its own equivalent: Justin: "The soft definition of AI-completeness is there's this fundamental primitive that's an AI task. But if I could solve this AI task in its full, broadest generality, it would solve any intelligence problem." "The classic example in LLMs is that next-token prediction is AI-complete... I think from Ilya: there's a mystery novel, the thing has to read the whole novel, and the final sentence is, 'And the killer was.' Predict the next token. You could basically frame any kind of intelligence task in terms of that." "So clearly next-token prediction is something people believe is AI-complete." "New-view prediction, this primitive that we have in Atlas, especially generative new-view prediction, is also AI-complete." "I want to have a world where Martin is writing a proof of the Riemann hypothesis on the blackboard, and then the camera pans over to the next whiteboard." Fei-Fei: "Evolution had to solve new-viewpoint prediction by making animals move. Nature gave animals eyes, but nature didn't give trees eyes. Why? Because when you move, you see a new viewpoint... We do believe very strongly that next-viewpoint prediction is the equivalent of next-token prediction." Justin Johnson Fei-Fei Li martin_casado

World Labs co-founders Justin Johnson and Dr. Fei-Fei Li say LLMs use next-token prediction, but spatial intelligence has its own equivalent: Justin: "The soft definition of AI-completeness is there's this fundamental primitive that's an AI task. But if I could solve this AI task in its full, broadest generality, it would solve any intelligence problem." "The classic example in LLMs is that next-token prediction is AI-complete... I think from Ilya: there's a mystery novel, the thing has to read the whole novel, and the final sentence is, 'And the killer was.' Predict the next token. You could basically frame any kind of intelligence task in terms of that." "So clearly next-token prediction is something people believe is AI-complete." "New-view prediction, this primitive that we have in Atlas, especially generative new-view prediction, is also AI-complete." "I want to have a world where Martin is writing a proof of the Riemann hypothesis on the blackboard, and then the camera pans over to the next whiteboard." Fei-Fei: "Evolution had to solve new-viewpoint prediction by making animals move. Nature gave animals eyes, but nature didn't give trees eyes. Why? Because when you move, you see a new viewpoint... We do believe very strongly that next-viewpoint prediction is the equivalent of next-token prediction." Justin Johnson Fei-Fei Li martin_casado

a16z

87,911 görüntüleme • 15 gün önce

Understanding Iliotibial Band Pathology: When discussing the iliotibial band, it's important to note that although its anatomical insertion is at Gerdy's tubercle, this isn't typically the site where pathology develops. Instead, problems often arise at the level of the lateral femoral condyle, particularly with repeated knee flexion. Long-distance runners** are a classic example. With ongoing flexion, the vastus lateralis muscle compresses the fat pad situated between the iliotibial band and the lateral femoral condyle. This compression leads to changes in the fat pad, often observed as echogenic and hyperaemic fat on imaging—hallmarks of iliotibial band compression syndrome. Not Just Friction—It's Compression: It’s crucial to clarify that this condition is not a friction syndrome, as is commonly believed. Rather, it is a *compression syndrome*, with the underlying mechanism being the direct pressure on the fat pad rather than friction between the band and bone. Understanding the precise location and mechanism of iliotibial band pathology helps us better diagnose and manage athletes and active individuals who present with lateral knee pain. Video Original Source: Sonography_Secrets You Tube

Understanding Iliotibial Band Pathology: When discussing the iliotibial band, it's important to note that although its anatomical insertion is at Gerdy's tubercle, this isn't typically the site where pathology develops. Instead, problems often arise at the level of the lateral femoral condyle, particularly with repeated knee flexion. Long-distance runners** are a classic example. With ongoing flexion, the vastus lateralis muscle compresses the fat pad situated between the iliotibial band and the lateral femoral condyle. This compression leads to changes in the fat pad, often observed as echogenic and hyperaemic fat on imaging—hallmarks of iliotibial band compression syndrome. Not Just Friction—It's Compression: It’s crucial to clarify that this condition is not a friction syndrome, as is commonly believed. Rather, it is a compression syndrome, with the underlying mechanism being the direct pressure on the fat pad rather than friction between the band and bone. Understanding the precise location and mechanism of iliotibial band pathology helps us better diagnose and manage athletes and active individuals who present with lateral knee pain. Video Original Source: Sonography_Secrets You Tube

Benoy Mathew

29,379 görüntüleme • 11 ay önce

Ilya Sutskever explains the 1% human training loop, and why fully autonomous AI is the wrong target "The thing you really want is for the human teachers that tell, that teach the AI, for them to collaborate with an AI." "You might want to think of it as being in a world where the human teachers do 1% of the world and the work and the AI do 99% of the work." "You don't want it to be 100% AI" "But you do want it to be a human-machine collaboration which teaches the next machine." Most teams are trying to remove the human from the loop. Ilya's design keeps a tiny human control surface and uses it to teach the next generation. The leverage is not zero humans. It is 1% human judgment steering 99% machine work. - Ilya Sutskever (Ilya Sutskever), AI researcher, on Dwarkesh Patel

Ilya Sutskever explains the 1% human training loop, and why fully autonomous AI is the wrong target "The thing you really want is for the human teachers that tell, that teach the AI, for them to collaborate with an AI." "You might want to think of it as being in a world where the human teachers do 1% of the world and the work and the AI do 99% of the work." "You don't want it to be 100% AI" "But you do want it to be a human-machine collaboration which teaches the next machine." Most teams are trying to remove the human from the loop. Ilya's design keeps a tiny human control surface and uses it to teach the next generation. The leverage is not zero humans. It is 1% human judgment steering 99% machine work. - Ilya Sutskever (Ilya Sutskever), AI researcher, on Dwarkesh Patel

Karl Mehta

30,651 görüntüleme • 2 ay önce

(flashing lights warning!!!!) 'Artifacts' is my tribute to the outsidery and computery aesthetics of Flash animation - viewable ONLY in flash (link in the next tweet) because as you can see, video compression *completely* nukes it!

(flashing lights warning!!!!) 'Artifacts' is my tribute to the outsidery and computery aesthetics of Flash animation - viewable ONLY in flash (link in the next tweet) because as you can see, video compression completely nukes it!

secret guy

72,555 görüntüleme • 3 yıl önce

WHOS READY TO RUN IT BACK TURBO?! Introducing FUTURE DAO! So we got DAOs ✅ We got Prediction markets ✅ We got Ai ✅ We got meme-coins✅ We got a pre sale✅ Prediction markets have been a bit boring to me TBH, I WANT TO HIT 100Xs not like do 10hrs of research on Greenland politics or something. So I created this DAO to give people some leverage, and make it fun. Basically how it works, is we do a pre sale, people get tokens, the rest of the token supply goes into the DAO. Like a classic DAOs fun but you can redeem the AUM at any time,(but you incur a penalty) . You have the AUM, and our proprietary Ai does the trading for you. Just hold the token and watch it go up(hopefully), the Ai trades with guidance from Token holder votes. The token holders also get to vote/guide the Ai into outcomes they think will benefit the DAO/token. AUM Goes up. Token Goes up. Everyone happy. Or AUM flat/Down. Token Goes down. People Redeem and get % of their money back. better than zeroing out on meme coins. The token will all be launched via a semi-gated pre sale, And using my own method of launching to showcase how its done and to ensure there is a good group of "seed DAO" members to help the Ai. it will run 8 weeks, and people can decide to shut it down and get $$ back. or keep it going forever. end of the day this is a meme coin guys, but I think its a cool experiment.

WHOS READY TO RUN IT BACK TURBO?! Introducing FUTURE DAO! So we got DAOs ✅ We got Prediction markets ✅ We got Ai ✅ We got meme-coins✅ We got a pre sale✅ Prediction markets have been a bit boring to me TBH, I WANT TO HIT 100Xs not like do 10hrs of research on Greenland politics or something. So I created this DAO to give people some leverage, and make it fun. Basically how it works, is we do a pre sale, people get tokens, the rest of the token supply goes into the DAO. Like a classic DAOs fun but you can redeem the AUM at any time,(but you incur a penalty) . You have the AUM, and our proprietary Ai does the trading for you. Just hold the token and watch it go up(hopefully), the Ai trades with guidance from Token holder votes. The token holders also get to vote/guide the Ai into outcomes they think will benefit the DAO/token. AUM Goes up. Token Goes up. Everyone happy. Or AUM flat/Down. Token Goes down. People Redeem and get % of their money back. better than zeroing out on meme coins. The token will all be launched via a semi-gated pre sale, And using my own method of launching to showcase how its done and to ensure there is a good group of "seed DAO" members to help the Ai. it will run 8 weeks, and people can decide to shut it down and get $$ back. or keep it going forever. end of the day this is a meme coin guys, but I think its a cool experiment.

Skely

47,630 görüntüleme • 8 ay önce

"When you go to spaces like VR or BCI, everybody knows each other, and that means there's a high density high quality founders, and whoever makes it work in the bear market -- they are going to crush it when the market turns better." Felix Hartmann explains why some of the most compelling asymmetric opportunities lie in ecosystems like VR and brain-computer interfaces. Watch the full episode of Angels to Allocators:

"When you go to spaces like VR or BCI, everybody knows each other, and that means there's a high density high quality founders, and whoever makes it work in the bear market -- they are going to crush it when the market turns better." Felix Hartmann explains why some of the most compelling asymmetric opportunities lie in ecosystems like VR and brain-computer interfaces. Watch the full episode of Angels to Allocators:

Hartmann Capital

378,971 görüntüleme • 1 yıl önce

Congratulations to the Florida Highway Patrol on being voted America’s best looking cruiser for the 4th year in a row! Huge thanks to everyone who voted and helped make it happen. And to Michigan… we know the Mustang has roots there, but it looks like it found a better home in Florida🌴 Better luck next year. 😉 #GradyJudd #FHP

Congratulations to the Florida Highway Patrol on being voted America’s best looking cruiser for the 4th year in a row! Huge thanks to everyone who voted and helped make it happen. And to Michigan… we know the Mustang has roots there, but it looks like it found a better home in Florida🌴 Better luck next year. 😉 #GradyJudd #FHP

Polk County Sheriff 🚔 Grady Judd

14,279 görüntüleme • 2 ay önce

Well that’s a better end to the week, 11 of last weeks injured calves plus some newbies back in their pen and by the looks of it they’re as happy as me about it. Just 7 left in intensive care for the next week while their wounds heal. Happy Friday, be like a calf on clean straw!

Well that’s a better end to the week, 11 of last weeks injured calves plus some newbies back in their pen and by the looks of it they’re as happy as me about it. Just 7 left in intensive care for the next week while their wounds heal. Happy Friday, be like a calf on clean straw!

Phil Latham

15,454 görüntüleme • 2 yıl önce

Bucke says he is tired of not being invited to events like Pro-Am but people who said slurs on stream are.😬 "How many people in the Pro-Am have said the N-Word? I'm better off in my career just saying it. At some point if I just say it and apologise I would do better"

Bucke says he is tired of not being invited to events like Pro-Am but people who said slurs on stream are.😬 "How many people in the Pro-Am have said the N-Word? I'm better off in my career just saying it. At some point if I just say it and apologise I would do better"

Stack

78,841 görüntüleme • 2 ay önce

sooyoung trying to convince yuri to yield the center position to her 😂😭 yuri: but you’re too tall sooyoung: because i’m taller, the picture of me in the center brings the stable feeling sometimes hyoyeon: when you’re in the center, it’s like swearing 😭 people just feel bad about it sooyoung: but if yuri’s in the center, it looks like do-re-mi, it feels unstable and like a downhill… yuri: still better than looking like a middle finger ^^ *both yuri and hyo means the 3 of them will look like this ㅗ (similar shape of a middle finger) if syoo is in the center 😭 MY POOR SOOYOUNG

sooyoung trying to convince yuri to yield the center position to her 😂😭 yuri: but you’re too tall sooyoung: because i’m taller, the picture of me in the center brings the stable feeling sometimes hyoyeon: when you’re in the center, it’s like swearing 😭 people just feel bad about it sooyoung: but if yuri’s in the center, it looks like do-re-mi, it feels unstable and like a downhill… yuri: still better than looking like a middle finger ^^ *both yuri and hyo means the 3 of them will look like this ㅗ (similar shape of a middle finger) if syoo is in the center 😭 MY POOR SOOYOUNG

ؘ

150,292 görüntüleme • 4 ay önce

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang

MTS

100,519 görüntüleme • 2 ay önce

AI EXPLAINS WHAT IT MEANS TO BE CHOSEN NOT EVERYONE IS CHOSEN & THATS WHY THEY CANT SEE THE TRUTH, OR EVEN WANT TO NEITHER DOES THEIR VIBRATION RESONATE ON THE SAME LEVEL AS A CHOSEN ONE WE. ARE. NOT. THE. SAME.

AI EXPLAINS WHAT IT MEANS TO BE CHOSEN NOT EVERYONE IS CHOSEN & THATS WHY THEY CANT SEE THE TRUTH, OR EVEN WANT TO NEITHER DOES THEIR VIBRATION RESONATE ON THE SAME LEVEL AS A CHOSEN ONE WE. ARE. NOT. THE. SAME.

Lesgo Brando

169,221 görüntüleme • 7 ay önce