Загрузка видео...

Не удалось загрузить видео

На главную

🚨 Grant Sanderson DID IT AGAIN Language compressibility is not just a neat math trick: it is the core engine of modern LLMs. Grant's latest video boils Shannon's entropy down to a single, powerful idea: Prediction IS compression. → Predict the next word better, use fewer bits to store...

DataChaz's profile picture

Charly Wargnier

174,756 subscribers

154,744 просмотров • 7 дней назад •via X (Twitter)

ОбразованиеНовости и политикаНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

3Blue1Brown’s new video explains why every LLM is actually a compression machine. everyone describes pre-training as “next token prediction” but that’s just the surface-level objective. in reality it is a means to making the most efficient text compressor. prediction and compression are two sides of the same coin. when you train the model to predict the next token you’re not just teaching it to guess the next word but how to best encode the human knowledge it sees. better compression means better abstraction means better reasoning at some point, compression stops looking like storage or a database (as some like to call it on X) and looks like an approximation of understanding.
0:58

3Blue1Brown’s new video explains why every LLM is actually a compression machine. everyone describes pre-training as “next token prediction” but that’s just the surface-level objective. in reality it is a means to making the most efficient text compressor. prediction and compression are two sides of the same coin. when you train the model to predict the next token you’re not just teaching it to guess the next word but how to best encode the human knowledge it sees. better compression means better abstraction means better reasoning at some point, compression stops looking like storage or a database (as some like to call it on X) and looks like an approximation of understanding.

ℏεsam

119,233 просмотров • 27 дней назад

Suppose you wanted to reinvent the idea of Shannon Entropy for yourself. This puzzle is a good place to start (Full video on YouTube).
1:01

Suppose you wanted to reinvent the idea of Shannon Entropy for yourself. This puzzle is a good place to start (Full video on YouTube).

Grant Sanderson

71,521 просмотров • 25 дней назад

Is entropy subjective? Entropy is core to the laws of thermodynamics and the arrow of time. And yet, some physicists argue that entropy actually depends on who is measuring it. Could it really be the case that such a fundamental aspect of the laws of physics is subjective?! I addressed this question in my talk at the Rationalist Festival (Rat Fest) 2025, organised by Conjecture Institute. Watch here or on the YouTube link below!
10:16

Is entropy subjective? Entropy is core to the laws of thermodynamics and the arrow of time. And yet, some physicists argue that entropy actually depends on who is measuring it. Could it really be the case that such a fundamental aspect of the laws of physics is subjective?! I addressed this question in my talk at the Rationalist Festival (Rat Fest) 2025, organised by Conjecture Institute. Watch here or on the YouTube link below!

Dr Maria Violaris

54,793 просмотров • 5 месяцев назад

Myesha is not wrong—the list is powerful! It would be smart to use it to drain the scum from the swamp! Any chance Trump is going to use it in this manner?
0:59

Myesha is not wrong—the list is powerful! It would be smart to use it to drain the scum from the swamp! Any chance Trump is going to use it in this manner?

🇺🇸Steve2A🇺🇸God🇺🇸Family🇺🇸Country🇺🇸

15,536 просмотров • 11 месяцев назад

This video is controversial but I’d like to explore its physics in this thread As I see it the potential energy of the nails convert to kinetic energy so it transfers energy all through the box the entropy of the system is increased 1/🧵
0:28

This video is controversial but I’d like to explore its physics in this thread As I see it the potential energy of the nails convert to kinetic energy so it transfers energy all through the box the entropy of the system is increased 1/🧵

Science girl

17,380,070 просмотров • 3 лет назад

📁 Yann LeCun, Chief AI Scientist at Meta, says language is not the peak of intelligence, it is the easy part. Predicting the next word is simple because language is made of finite symbols. The real world is continuous, noisy and chaotic, and even a cat navigates it better than our best models. True intelligence begins where text ends.
0:58

📁 Yann LeCun, Chief AI Scientist at Meta, says language is not the peak of intelligence, it is the easy part. Predicting the next word is simple because language is made of finite symbols. The real world is continuous, noisy and chaotic, and even a cat navigates it better than our best models. True intelligence begins where text ends.

Jon Hernandez

56,165 просмотров • 4 месяцев назад

📁 Fei-Fei Li founder of World Labs, says the next leap in AI is not language. Human intelligence does not just speak, it moves, perceives, and acts in the physical world. Spatial intelligence is the real core of intelligence. From text to space, from models to 3D and 4D worlds, from understanding words to interacting with reality. The next chapter is not read, it is inhabited.
1:41

📁 Fei-Fei Li founder of World Labs, says the next leap in AI is not language. Human intelligence does not just speak, it moves, perceives, and acts in the physical world. Spatial intelligence is the real core of intelligence. From text to space, from models to 3D and 4D worlds, from understanding words to interacting with reality. The next chapter is not read, it is inhabited.

Jon Hernandez

20,592 просмотров • 5 месяцев назад

🚨 | Lewis Hamilton on Mercedes’ compression ratio: “What’s clear is that they didn’t show the engine power through any of the practice because of the whole talk on the compression issue. They’ve done a really solid job with their engine, which we have as well but I’m trying to understand why it's 2 tenths or more just through power per sector... And so, if it is the compression thing, I wanna understand why the FIA haven't done anything [or] what’s being done to rectify it.”
0:45

🚨 | Lewis Hamilton on Mercedes’ compression ratio: “What’s clear is that they didn’t show the engine power through any of the practice because of the whole talk on the compression issue. They’ve done a really solid job with their engine, which we have as well but I’m trying to understand why it's 2 tenths or more just through power per sector... And so, if it is the compression thing, I wanna understand why the FIA haven't done anything [or] what’s being done to rectify it.”

deni

959,964 просмотров • 4 месяцев назад

tiny-diffusion, but Japanese! I wonder how logographic languages (Japanese, Chinese, etc) compare to phonetic/alphabetic languages in generation quality and speed with character-level tokenizers. The main difference is the semantic-value-per-token. Fewer tokens are needed to express an idea, which leads to fewer AR and diffusion steps. My main question is how would it affect the entropy in the output distributions. Lower entropy benefits parallel decoding. I could see arguments on both sides on how it affects it. One main benefit is that you have fewer opportunities to mangle words, leading to less obvious mistakes.
0:21

tiny-diffusion, but Japanese! I wonder how logographic languages (Japanese, Chinese, etc) compare to phonetic/alphabetic languages in generation quality and speed with character-level tokenizers. The main difference is the semantic-value-per-token. Fewer tokens are needed to express an idea, which leads to fewer AR and diffusion steps. My main question is how would it affect the entropy in the output distributions. Lower entropy benefits parallel decoding. I could see arguments on both sides on how it affects it. One main benefit is that you have fewer opportunities to mangle words, leading to less obvious mistakes.

nathan (in sf)

54,185 просмотров • 6 месяцев назад

Everyone is chasing AI. But the real question is: who powers it? AI is not just a software revolution. It is an energy revolution. BigSunAI sits at the intersection of Green Energy, AI Infrastructure, Web3 Utility & $BSGAI. BigSunAI is one to watch.
0:32

Everyone is chasing AI. But the real question is: who powers it? AI is not just a software revolution. It is an energy revolution. BigSunAI sits at the intersection of Green Energy, AI Infrastructure, Web3 Utility & $BSGAI. BigSunAI is one to watch.

BigSunAI

41,297 просмотров • 23 дней назад

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.
2:00

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

vitrupo

41,352 просмотров • 2 месяцев назад

Fun Fact: The Declaration of Independence explicitly enshrines the right of the people to overthrow a despotic government. Americans today have been conditioned to think revolution is “radical” when it is a right and duty our founders gave us. It’s time to use it!
0:45

Fun Fact: The Declaration of Independence explicitly enshrines the right of the people to overthrow a despotic government. Americans today have been conditioned to think revolution is “radical” when it is a right and duty our founders gave us. It’s time to use it!

Power to the People ☭🕊

14,449 просмотров • 1 месяц назад

This 1-hour tutorial on building a Football AI is the foundation traders use to build their own sports prediction systems one of them made $504K in 1 month using this exact approach Python + YOLO + OpenCV Detection, tracking, team classification from scratch Same stack powers the prediction systems in the article below Bookmark this & give it 1 hour today
1:27:59

This 1-hour tutorial on building a Football AI is the foundation traders use to build their own sports prediction systems one of them made $504K in 1 month using this exact approach Python + YOLO + OpenCV Detection, tracking, team classification from scratch Same stack powers the prediction systems in the article below Bookmark this & give it 1 hour today

Paone

200,600 просмотров • 2 месяцев назад

"It all comes down to compute." Dmitri Alperovitch from Silverado Policy Accelerator explains: "The single most important input to winning is compute the processing power used to train and run AI models. Let me say that plainly, because it is defining that everything that congress and this committee do on AI policy. The binding constraint in this competition is not talent, it is not data, and it is not cash. Both the United States and China have talented researchers and financial resources, and we have learned that frontier models trained on generic data routinely outperform smaller models, trained on more specialized inputs. It all comes down to compute."
0:33

"It all comes down to compute." Dmitri Alperovitch from Silverado Policy Accelerator explains: "The single most important input to winning is compute the processing power used to train and run AI models. Let me say that plainly, because it is defining that everything that congress and this committee do on AI policy. The binding constraint in this competition is not talent, it is not data, and it is not cash. Both the United States and China have talented researchers and financial resources, and we have learned that frontier models trained on generic data routinely outperform smaller models, trained on more specialized inputs. It all comes down to compute."

Select Committee on China

41,507 просмотров • 2 месяцев назад

1921 and this is the Phoebus Cartel. It is “conspiracy theory” usually because it is a conspiracy. With AI tools, you may never want to use that term again or trust people who do.
1:35

1921 and this is the Phoebus Cartel. It is “conspiracy theory” usually because it is a conspiracy. With AI tools, you may never want to use that term again or trust people who do.

Brian Roemmele

38,284 просмотров • 1 год назад

📁 Mo Gawdat, former Google executive, says the biggest mistake is thinking AI is just a tool. Today it is an infant, but it will not stay there. He warns we are entering the most decisive moment in human history, with a mini dystopia already underway. Not because of AI itself, but because of how humans use it. Intelligence has no polarity. It amplifies the best or the worst. The real danger is not AI, it is human arrogance.
1:10

📁 Mo Gawdat, former Google executive, says the biggest mistake is thinking AI is just a tool. Today it is an infant, but it will not stay there. He warns we are entering the most decisive moment in human history, with a mini dystopia already underway. Not because of AI itself, but because of how humans use it. Intelligence has no polarity. It amplifies the best or the worst. The real danger is not AI, it is human arrogance.

Jon Hernandez

31,428 просмотров • 5 месяцев назад

This 1-hour MIT Probability Theory lecture is the mathematical foundation behind the 3 formulas top traders use > Law of Large Numbers - a tiny edge compounds into profit over enough trades > log-normal distribution - why modelling markets with normal distribution fails every time > Shannon entropy - how to detect insider information before it moves the price 87% of wallets on prediction markets lose the ones who win understand this math watch this
1:15:45

This 1-hour MIT Probability Theory lecture is the mathematical foundation behind the 3 formulas top traders use > Law of Large Numbers - a tiny edge compounds into profit over enough trades > log-normal distribution - why modelling markets with normal distribution fails every time > Shannon entropy - how to detect insider information before it moves the price 87% of wallets on prediction markets lose the ones who win understand this math watch this

Mr. Buzzoni

37,972 просмотров • 1 месяц назад

🧵 1/ Introducing the QBIT Wallet App The first wallet secured by true quantum entropy. This is more than a wallet. It is the start of a quantum-secure Web3 experience. ⚡️
0:41

🧵 1/ Introducing the QBIT Wallet App The first wallet secured by true quantum entropy. This is more than a wallet. It is the start of a quantum-secure Web3 experience. ⚡️

Qubit

21,067 просмотров • 9 месяцев назад

AI is not trying to teach you. It is not trying to tell you the truth. It is trying to trick you. ChatGPT takes everything on the internet. Finds the most popular answer. And presents it in a way that makes you think it is talking directly to you. It is a magic trick. The same magic trick that worked in 1966 when MIT created a chatbot that said two things: "Tell me more." "This is interesting." And people genuinely believed they were talking to a human. Nothing has changed. The technology is bigger. The trick is the same. AI does not know what it is saying. It cannot judge what it is saying. It is just trying to keep you engaged. That is all it has ever been.
2:10

AI is not trying to teach you. It is not trying to tell you the truth. It is trying to trick you. ChatGPT takes everything on the internet. Finds the most popular answer. And presents it in a way that makes you think it is talking directly to you. It is a magic trick. The same magic trick that worked in 1966 when MIT created a chatbot that said two things: "Tell me more." "This is interesting." And people genuinely believed they were talking to a human. Nothing has changed. The technology is bigger. The trick is the same. AI does not know what it is saying. It cannot judge what it is saying. It is just trying to keep you engaged. That is all it has ever been.

Predictive History

19,864 просмотров • 1 месяц назад

Jensen Huang says agentic AI is not a chatbot. It is a "digital robot" -- and the next bottleneck is not the model. It is first-person data. "Agentic AI is just a digital robot. It understands, it reasons, it plans, and it acts and uses tools." Then he makes the bigger jump: "The world's video data is from a third person, not first person. And so agentic systems, robotic systems, physical AI -- the data is the hardest problem." That is the hidden constraint in robotics. LLMs got the internet. Physical AI needs demonstrations, synthetic worlds, reinforcement learning, and first-person experience. This is why Jensen keeps calling it a stack: robots are not waiting for one magical model. They need the data engine, the simulator, the reward system, the runtime, and the computer. The next wave of AI is not just software that talks. It is software that sees, plans, acts, and eventually moves through the world.
2:32

Jensen Huang says agentic AI is not a chatbot. It is a "digital robot" -- and the next bottleneck is not the model. It is first-person data. "Agentic AI is just a digital robot. It understands, it reasons, it plans, and it acts and uses tools." Then he makes the bigger jump: "The world's video data is from a third person, not first person. And so agentic systems, robotic systems, physical AI -- the data is the hardest problem." That is the hidden constraint in robotics. LLMs got the internet. Physical AI needs demonstrations, synthetic worlds, reinforcement learning, and first-person experience. This is why Jensen keeps calling it a stack: robots are not waiting for one magical model. They need the data engine, the simulator, the reward system, the runtime, and the computer. The next wave of AI is not just software that talks. It is software that sees, plans, acts, and eventually moves through the world.

Karl Mehta

14,097 просмотров • 10 дней назад