正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

3Blue1Brown’s new video explains why every LLM is actually a compression machine. everyone describes pre-training as “next token prediction” but that’s just the surface-level objective. in reality it is a means to making the most efficient text compressor. prediction and compression are two sides of the same coin. when... show more

ℏεsam

83,662 subscribers

119,961 次观看 • 1 个月前 •via X (Twitter)

教育新闻政治科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

🚨 ANOTHER MASTERCLASS FROM Grant Sanderson The compressibility of language isn’t just a math curiosity, it’s the hidden engine behind every LLM you use. Grant’s new video reframes Shannon’s entropy through one elegant lens: Prediction IS compression. → The better you predict the next word, the fewer bits you need to store it → Shannon measured English at ~1 bit per character: astonishingly compressible → This is exactly what GPT-style models optimize → Intelligence, in this framing, is compression FUN FACT: Von Neumann told Shannon to name it “entropy” because nobody truly understands it anyway 😄 Decades later, that same concept became the bedrock of modern AI. Deep-dive resources in the 🧵 ↓

🚨 ANOTHER MASTERCLASS FROM Grant Sanderson The compressibility of language isn’t just a math curiosity, it’s the hidden engine behind every LLM you use. Grant’s new video reframes Shannon’s entropy through one elegant lens: Prediction IS compression. → The better you predict the next word, the fewer bits you need to store it → Shannon measured English at ~1 bit per character: astonishingly compressible → This is exactly what GPT-style models optimize → Intelligence, in this framing, is compression FUN FACT: Von Neumann told Shannon to name it “entropy” because nobody truly understands it anyway 😄 Decades later, that same concept became the bedrock of modern AI. Deep-dive resources in the 🧵 ↓

Charly Wargnier

319,107 次观看 • 1 个月前

🚨 Grant Sanderson DID IT AGAIN Language compressibility is not just a neat math trick: it is the core engine of modern LLMs. Grant's latest video boils Shannon's entropy down to a single, powerful idea: Prediction IS compression. → Predict the next word better, use fewer bits to store it → Shannon found English is astonishingly compressible (~1 bit per character) → This is the exact mechanism GPT models run on → Under this framing, intelligence equals compression FUN FACT: Von Neumann told Shannon to use the term "entropy" because no one really understands it. Today, it powers the AI revolution. Deep-dive resources in the 🧵↓

🚨 Grant Sanderson DID IT AGAIN Language compressibility is not just a neat math trick: it is the core engine of modern LLMs. Grant's latest video boils Shannon's entropy down to a single, powerful idea: Prediction IS compression. → Predict the next word better, use fewer bits to store it → Shannon found English is astonishingly compressible (~1 bit per character) → This is the exact mechanism GPT models run on → Under this framing, intelligence equals compression FUN FACT: Von Neumann told Shannon to use the term "entropy" because no one really understands it. Today, it powers the AI revolution. Deep-dive resources in the 🧵↓

Charly Wargnier

160,113 次观看 • 1 个月前

Ilya Sutskever, co-founder and chief scientist of OpenAI: "The better a neural network can predict the next word, the more it understands it." Everyone calls it autocomplete. Ilya explained why that is a comforting lie. To guess the final word of a murder mystery, the name of the killer, the model has to reason through every clue in the book. Prediction that good is not statistics. It is understanding. Then he said the part that should stop you. The text these models train on is "a projection of the world," and to compress it they learn more and more about people, "the human condition, their hopes, dreams, and motivations." The machine is not memorizing our words. It is quietly building a model of us. That is the whole premise of the fear. The machine will never hate you. It understands you far better than that. The frightening variable was never the intelligence. It was the aim. I wrote the full story: the one fear about AI everyone gets wrong, and the one already here.

Ilya Sutskever, co-founder and chief scientist of OpenAI: "The better a neural network can predict the next word, the more it understands it." Everyone calls it autocomplete. Ilya explained why that is a comforting lie. To guess the final word of a murder mystery, the name of the killer, the model has to reason through every clue in the book. Prediction that good is not statistics. It is understanding. Then he said the part that should stop you. The text these models train on is "a projection of the world," and to compress it they learn more and more about people, "the human condition, their hopes, dreams, and motivations." The machine is not memorizing our words. It is quietly building a model of us. That is the whole premise of the fear. The machine will never hate you. It understands you far better than that. The frightening variable was never the intelligence. It was the aim. I wrote the full story: the one fear about AI everyone gets wrong, and the one already here.

Casper

91,161 次观看 • 8 天前

🚨| Sir Lewis Hamilton on Mercedes' engine: "What's clear is that they didn't show the engine power through any of the practice, because there's the whole talk of compression ratio, and they've obviously done a really solid job with their engine, which we have as well, but it would be interesting to... I want to understand why it's like two-tenths or more just through power, per sector. And so, if it is a compression thing, I want to understand why the FIA haven't done anything, what's being done to rectify it, but if it's not, and it's just pure [ped], then we have to do a better job." #F1 || #AustralianGP

🚨| Sir Lewis Hamilton on Mercedes' engine: "What's clear is that they didn't show the engine power through any of the practice, because there's the whole talk of compression ratio, and they've obviously done a really solid job with their engine, which we have as well, but it would be interesting to... I want to understand why it's like two-tenths or more just through power, per sector. And so, if it is a compression thing, I want to understand why the FIA haven't done anything, what's being done to rectify it, but if it's not, and it's just pure [ped], then we have to do a better job." #F1 || #AustralianGP

sim

38,390 次观看 • 4 个月前

🚨 | Lewis Hamilton: “If it is the compression thing, I wanna understand why the FIA haven’t done anything, what’s been done to rectify it but if it’s not and it’s just pure pace, then we have to do a better job.”

🚨 | Lewis Hamilton: “If it is the compression thing, I wanna understand why the FIA haven’t done anything, what’s been done to rectify it but if it’s not and it’s just pure pace, then we have to do a better job.”

La Gazzetta Ferrari

134,199 次观看 • 4 个月前

🚨 | Lewis Hamilton on Mercedes’ compression ratio: “What’s clear is that they didn’t show the engine power through any of the practice because of the whole talk on the compression issue. They’ve done a really solid job with their engine, which we have as well but I’m trying to understand why it's 2 tenths or more just through power per sector... And so, if it is the compression thing, I wanna understand why the FIA haven't done anything [or] what’s being done to rectify it.”

🚨 | Lewis Hamilton on Mercedes’ compression ratio: “What’s clear is that they didn’t show the engine power through any of the practice because of the whole talk on the compression issue. They’ve done a really solid job with their engine, which we have as well but I’m trying to understand why it's 2 tenths or more just through power per sector... And so, if it is the compression thing, I wanna understand why the FIA haven't done anything [or] what’s being done to rectify it.”

deni

960,066 次观看 • 4 个月前

Mr Shift: What's the true innovation with prediction markets? Kalshi CEO: There's many ways to answer that question. Um there's the Kalshi piece of it, but then there's the prediction market piece of it, which are not exactly the same. “But the... the true innovation in prediction markets, I think, is this notion, like... and, and I think it has been built through a lot of literature, and then I think the prediction markets have come in our generation, like Kalshi and some of the others have proven it” “The best way to answer like all these most pressing questions is like you have to ask the crowd”

Mr Shift: What's the true innovation with prediction markets? Kalshi CEO: There's many ways to answer that question. Um there's the Kalshi piece of it, but then there's the prediction market piece of it, which are not exactly the same. “But the... the true innovation in prediction markets, I think, is this notion, like... and, and I think it has been built through a lot of literature, and then I think the prediction markets have come in our generation, like Kalshi and some of the others have proven it” “The best way to answer like all these most pressing questions is like you have to ask the crowd”

ShaDow | Clipur.com

23,860 次观看 • 1 个月前

Understanding Iliotibial Band Pathology: When discussing the iliotibial band, it's important to note that although its anatomical insertion is at Gerdy's tubercle, this isn't typically the site where pathology develops. Instead, problems often arise at the level of the lateral femoral condyle, particularly with repeated knee flexion. Long-distance runners** are a classic example. With ongoing flexion, the vastus lateralis muscle compresses the fat pad situated between the iliotibial band and the lateral femoral condyle. This compression leads to changes in the fat pad, often observed as echogenic and hyperaemic fat on imaging—hallmarks of iliotibial band compression syndrome. Not Just Friction—It's Compression: It’s crucial to clarify that this condition is not a friction syndrome, as is commonly believed. Rather, it is a *compression syndrome*, with the underlying mechanism being the direct pressure on the fat pad rather than friction between the band and bone. Understanding the precise location and mechanism of iliotibial band pathology helps us better diagnose and manage athletes and active individuals who present with lateral knee pain. Video Original Source: Sonography_Secrets You Tube

Understanding Iliotibial Band Pathology: When discussing the iliotibial band, it's important to note that although its anatomical insertion is at Gerdy's tubercle, this isn't typically the site where pathology develops. Instead, problems often arise at the level of the lateral femoral condyle, particularly with repeated knee flexion. Long-distance runners** are a classic example. With ongoing flexion, the vastus lateralis muscle compresses the fat pad situated between the iliotibial band and the lateral femoral condyle. This compression leads to changes in the fat pad, often observed as echogenic and hyperaemic fat on imaging—hallmarks of iliotibial band compression syndrome. Not Just Friction—It's Compression: It’s crucial to clarify that this condition is not a friction syndrome, as is commonly believed. Rather, it is a compression syndrome, with the underlying mechanism being the direct pressure on the fat pad rather than friction between the band and bone. Understanding the precise location and mechanism of iliotibial band pathology helps us better diagnose and manage athletes and active individuals who present with lateral knee pain. Video Original Source: Sonography_Secrets You Tube

Benoy Mathew

29,379 次观看 • 9 个月前

Ilya Sutskever explains the 1% human training loop, and why fully autonomous AI is the wrong target "The thing you really want is for the human teachers that tell, that teach the AI, for them to collaborate with an AI." "You might want to think of it as being in a world where the human teachers do 1% of the world and the work and the AI do 99% of the work." "You don't want it to be 100% AI" "But you do want it to be a human-machine collaboration which teaches the next machine." Most teams are trying to remove the human from the loop. Ilya's design keeps a tiny human control surface and uses it to teach the next generation. The leverage is not zero humans. It is 1% human judgment steering 99% machine work. - Ilya Sutskever (Ilya Sutskever), AI researcher, on Dwarkesh Patel

Ilya Sutskever explains the 1% human training loop, and why fully autonomous AI is the wrong target "The thing you really want is for the human teachers that tell, that teach the AI, for them to collaborate with an AI." "You might want to think of it as being in a world where the human teachers do 1% of the world and the work and the AI do 99% of the work." "You don't want it to be 100% AI" "But you do want it to be a human-machine collaboration which teaches the next machine." Most teams are trying to remove the human from the loop. Ilya's design keeps a tiny human control surface and uses it to teach the next generation. The leverage is not zero humans. It is 1% human judgment steering 99% machine work. - Ilya Sutskever (Ilya Sutskever), AI researcher, on Dwarkesh Patel

Karl Mehta

30,546 次观看 • 12 天前

(flashing lights warning!!!!) 'Artifacts' is my tribute to the outsidery and computery aesthetics of Flash animation - viewable ONLY in flash (link in the next tweet) because as you can see, video compression *completely* nukes it!

(flashing lights warning!!!!) 'Artifacts' is my tribute to the outsidery and computery aesthetics of Flash animation - viewable ONLY in flash (link in the next tweet) because as you can see, video compression completely nukes it!

secret guy

72,513 次观看 • 3 年前

"When you go to spaces like VR or BCI, everybody knows each other, and that means there's a high density high quality founders, and whoever makes it work in the bear market -- they are going to crush it when the market turns better." Felix Hartmann explains why some of the most compelling asymmetric opportunities lie in ecosystems like VR and brain-computer interfaces. Watch the full episode of Angels to Allocators:

"When you go to spaces like VR or BCI, everybody knows each other, and that means there's a high density high quality founders, and whoever makes it work in the bear market -- they are going to crush it when the market turns better." Felix Hartmann explains why some of the most compelling asymmetric opportunities lie in ecosystems like VR and brain-computer interfaces. Watch the full episode of Angels to Allocators:

Hartmann Capital

378,971 次观看 • 11 个月前

WHOS READY TO RUN IT BACK TURBO?! Introducing FUTURE DAO! So we got DAOs ✅ We got Prediction markets ✅ We got Ai ✅ We got meme-coins✅ We got a pre sale✅ Prediction markets have been a bit boring to me TBH, I WANT TO HIT 100Xs not like do 10hrs of research on Greenland politics or something. So I created this DAO to give people some leverage, and make it fun. Basically how it works, is we do a pre sale, people get tokens, the rest of the token supply goes into the DAO. Like a classic DAOs fun but you can redeem the AUM at any time,(but you incur a penalty) . You have the AUM, and our proprietary Ai does the trading for you. Just hold the token and watch it go up(hopefully), the Ai trades with guidance from Token holder votes. The token holders also get to vote/guide the Ai into outcomes they think will benefit the DAO/token. AUM Goes up. Token Goes up. Everyone happy. Or AUM flat/Down. Token Goes down. People Redeem and get % of their money back. better than zeroing out on meme coins. The token will all be launched via a semi-gated pre sale, And using my own method of launching to showcase how its done and to ensure there is a good group of "seed DAO" members to help the Ai. it will run 8 weeks, and people can decide to shut it down and get $$ back. or keep it going forever. end of the day this is a meme coin guys, but I think its a cool experiment.

WHOS READY TO RUN IT BACK TURBO?! Introducing FUTURE DAO! So we got DAOs ✅ We got Prediction markets ✅ We got Ai ✅ We got meme-coins✅ We got a pre sale✅ Prediction markets have been a bit boring to me TBH, I WANT TO HIT 100Xs not like do 10hrs of research on Greenland politics or something. So I created this DAO to give people some leverage, and make it fun. Basically how it works, is we do a pre sale, people get tokens, the rest of the token supply goes into the DAO. Like a classic DAOs fun but you can redeem the AUM at any time,(but you incur a penalty) . You have the AUM, and our proprietary Ai does the trading for you. Just hold the token and watch it go up(hopefully), the Ai trades with guidance from Token holder votes. The token holders also get to vote/guide the Ai into outcomes they think will benefit the DAO/token. AUM Goes up. Token Goes up. Everyone happy. Or AUM flat/Down. Token Goes down. People Redeem and get % of their money back. better than zeroing out on meme coins. The token will all be launched via a semi-gated pre sale, And using my own method of launching to showcase how its done and to ensure there is a good group of "seed DAO" members to help the Ai. it will run 8 weeks, and people can decide to shut it down and get $$ back. or keep it going forever. end of the day this is a meme coin guys, but I think its a cool experiment.

Skely

47,630 次观看 • 6 个月前

Congratulations to the Florida Highway Patrol on being voted America’s best looking cruiser for the 4th year in a row! Huge thanks to everyone who voted and helped make it happen. And to Michigan… we know the Mustang has roots there, but it looks like it found a better home in Florida🌴 Better luck next year. 😉 #GradyJudd #FHP

Congratulations to the Florida Highway Patrol on being voted America’s best looking cruiser for the 4th year in a row! Huge thanks to everyone who voted and helped make it happen. And to Michigan… we know the Mustang has roots there, but it looks like it found a better home in Florida🌴 Better luck next year. 😉 #GradyJudd #FHP

Polk County Sheriff 🚔 Grady Judd

14,273 次观看 • 15 天前

Well that’s a better end to the week, 11 of last weeks injured calves plus some newbies back in their pen and by the looks of it they’re as happy as me about it. Just 7 left in intensive care for the next week while their wounds heal. Happy Friday, be like a calf on clean straw!

Well that’s a better end to the week, 11 of last weeks injured calves plus some newbies back in their pen and by the looks of it they’re as happy as me about it. Just 7 left in intensive care for the next week while their wounds heal. Happy Friday, be like a calf on clean straw!

Phil Latham

15,454 次观看 • 1 年前

sooyoung trying to convince yuri to yield the center position to her 😂😭 yuri: but you’re too tall sooyoung: because i’m taller, the picture of me in the center brings the stable feeling sometimes hyoyeon: when you’re in the center, it’s like swearing 😭 people just feel bad about it sooyoung: but if yuri’s in the center, it looks like do-re-mi, it feels unstable and like a downhill… yuri: still better than looking like a middle finger ^^ *both yuri and hyo means the 3 of them will look like this ㅗ (similar shape of a middle finger) if syoo is in the center 😭 MY POOR SOOYOUNG

sooyoung trying to convince yuri to yield the center position to her 😂😭 yuri: but you’re too tall sooyoung: because i’m taller, the picture of me in the center brings the stable feeling sometimes hyoyeon: when you’re in the center, it’s like swearing 😭 people just feel bad about it sooyoung: but if yuri’s in the center, it looks like do-re-mi, it feels unstable and like a downhill… yuri: still better than looking like a middle finger ^^ *both yuri and hyo means the 3 of them will look like this ㅗ (similar shape of a middle finger) if syoo is in the center 😭 MY POOR SOOYOUNG

ؘ

150,095 次观看 • 2 个月前

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang

MTS

99,784 次观看 • 8 天前

AI EXPLAINS WHAT IT MEANS TO BE CHOSEN NOT EVERYONE IS CHOSEN & THATS WHY THEY CANT SEE THE TRUTH, OR EVEN WANT TO NEITHER DOES THEIR VIBRATION RESONATE ON THE SAME LEVEL AS A CHOSEN ONE WE. ARE. NOT. THE. SAME.

AI EXPLAINS WHAT IT MEANS TO BE CHOSEN NOT EVERYONE IS CHOSEN & THATS WHY THEY CANT SEE THE TRUTH, OR EVEN WANT TO NEITHER DOES THEIR VIBRATION RESONATE ON THE SAME LEVEL AS A CHOSEN ONE WE. ARE. NOT. THE. SAME.

Lesgo Brando

168,448 次观看 • 5 个月前

.Naval: Look at reality the way it is. Try to take yourself out of the equation. “Labels like pessimist, optimist, cynic, introvert, extrovert—these are very self limiting. Humans are very dynamic. There are times when you feel like being introverted. There are times when you feel like being extroverted. There are contexts in which you’ll be pessimistic. There are contexts in which you’ll be optimistic. Leave all those labels alone. It’s better just to look at the problem at hand. Look at reality the way it is. Try to take yourself out of the equation, in a sense. Obviously you’re involved. But motivated reasoning is the worst kind of reasoning. You’re not going to find truth through highly motivated reasoning. You have to be objective. And objective means trying to take yourself out of it as much as possible, or at least your personality out of it as much as possible. And so to the extent you run with this thick identity and personality, it’s going to cloud your judgment. It’s going to try and lock you into the past. If you say, ‘I’m a depressed, unhappy person—I’m going to be unhappy.’ That’s a way of locking yourself into your past. Even saying, ‘I have trauma, I have PTSD,’ yeah, you feel something—there are memories, there are flashes, there are occasional bad feelings. But don’t define yourself by it because then you’ll lock it into your identity and you’re just going to loop on it. It’s better to stay flexible because reality is always changing and you have to be able to adapt to it. Adaptation is also intelligence, adaptation is survival. Adaptation is kind of how you’re here. You’re here because you’re an adapter and your ancestors were adapters. So to adapt you’ll have to see things clearly and if you’re seeing them through your own identity, it’s going to cloud your judgment.”

.Naval: Look at reality the way it is. Try to take yourself out of the equation. “Labels like pessimist, optimist, cynic, introvert, extrovert—these are very self limiting. Humans are very dynamic. There are times when you feel like being introverted. There are times when you feel like being extroverted. There are contexts in which you’ll be pessimistic. There are contexts in which you’ll be optimistic. Leave all those labels alone. It’s better just to look at the problem at hand. Look at reality the way it is. Try to take yourself out of the equation, in a sense. Obviously you’re involved. But motivated reasoning is the worst kind of reasoning. You’re not going to find truth through highly motivated reasoning. You have to be objective. And objective means trying to take yourself out of it as much as possible, or at least your personality out of it as much as possible. And so to the extent you run with this thick identity and personality, it’s going to cloud your judgment. It’s going to try and lock you into the past. If you say, ‘I’m a depressed, unhappy person—I’m going to be unhappy.’ That’s a way of locking yourself into your past. Even saying, ‘I have trauma, I have PTSD,’ yeah, you feel something—there are memories, there are flashes, there are occasional bad feelings. But don’t define yourself by it because then you’ll lock it into your identity and you’re just going to loop on it. It’s better to stay flexible because reality is always changing and you have to be able to adapt to it. Adaptation is also intelligence, adaptation is survival. Adaptation is kind of how you’re here. You’re here because you’re an adapter and your ancestors were adapters. So to adapt you’ll have to see things clearly and if you’re seeing them through your own identity, it’s going to cloud your judgment.”

Arjun Khemani

251,978 次观看 • 1 年前

Threadguy explains why Apple will capture ALL of the value from AI “Models have become a commodity, we'll be able to just use the best model at any given time” "The more data AI has access to, the better it is. Everyone already has an iPhone, the next iPhone is just going to launch with some OpenClaw esque assistant utilising the best model plugged into it and its probably going to be the best AI experience you’ve ever had”

Threadguy explains why Apple will capture ALL of the value from AI “Models have become a commodity, we'll be able to just use the best model at any given time” "The more data AI has access to, the better it is. Everyone already has an iPhone, the next iPhone is just going to launch with some OpenClaw esque assistant utilising the best model plugged into it and its probably going to be the best AI experience you’ve ever had”

Jack

24,179 次观看 • 5 个月前

Emotions come and go—that’s the nature of being human. One moment you feel joy, the next, anger or sadness. That doesn’t mean something’s wrong with you. It means you’re alive. The key is not to resist what you feel… but to understand it. To sit with it. To learn from it without letting it define you. Even the wisest man has storms within— he’s just learned how to sail through them.

Emotions come and go—that’s the nature of being human. One moment you feel joy, the next, anger or sadness. That doesn’t mean something’s wrong with you. It means you’re alive. The key is not to resist what you feel… but to understand it. To sit with it. To learn from it without letting it define you. Even the wisest man has storms within— he’s just learned how to sail through them.

𖣂Prince𖣂

175,498 次观看 • 1 年前