Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

MIT just published a paper proving that scaling LLMs has a hard mathematical ceiling - and doubling the width to get there costs 6x more each time. A frontier model costs $300 million to train today. Double it once: $1.8 billion. Four more times: $390 billion for a single... training run. Even if energy became free, the error rate would still drop at exactly the same speed. The exponent is locked by the structure of human language itself. The mechanism is superposition. GPT-3 has 12,000 dimensions but over 50,000 concepts to store - so concepts like "elephant" and "balloon" end up pointing in almost the same direction in embedding space. When the model activates one, the other bleeds through. That bleed is the interference causing hallucinations. Doubling the width cuts interference in half - but once every concept gets its own direction, scaling buys exactly nothing. 7 minutes. Bookmark & watch today. The hard limit on LLM scaling that money cannot fix.show more

Eron

8,875 subscribers

48,615 просмотров • 6 дней назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля rocky

rocky5 дней назад

classic clickbait take on neural scaling. superposition noise is real at the representation layer, but hardware costs and hallucination rates in enterprise workflows aren't bounded by embedding dimensions

Фото профиля Artemis

Artemis6 дней назад

Not to worry, they're going to slow things down so we will never reach the limit. For all of humanity.

Фото профиля Tony Scott 🧄(🦆🐓🐵🧪🧬🪪)❌=↑🧄🧄🧄🥩🥚🧀↓👽👾🤖

Tony Scott 🧄(🦆🐓🐵🧪🧬🪪)❌=↑🧄🧄🧄🥩🥚🧀↓👽👾🤖5 дней назад

@grok explain this issue with vector weights

Фото профиля mark

mark5 дней назад

bro discovered pretraining scaling laws and called them a ceiling on intelligence. the paper proposes a mechanism for existing scaling laws. it doesn’t prove a hard ceiling and next-token loss isn’t a hallucination rate. did you read the paper or just generate the apocalypse?

Фото профиля JLivermore

JLivermore5 дней назад

@grok qual é o canal do vídeo acima?

Фото профиля Elis

Elis5 дней назад

“MIT just published” — May 2025. And where exactly is this “hard mathematical ceiling” proved? The paper models one source of representation loss, not a limit on LLM capabilities. The linked video makes the same unsupported leap. Did either of you actually read the paper?

Фото профиля sova prase

sova prase5 дней назад

So we can double at least four times from now 2x2x2x2x… I’m fine with that. It’s enough make it 16 times larger

Фото профиля Singulating Entropic Abyss

Singulating Entropic Abyss6 дней назад

In the end, an inexpensive after market modification, like an electrin tunneling device or a resonant divergence stack of random number generators... will turn GPT-2 into beyond super intelligent...

Фото профиля Degribeauval🎩

Degribeauval🎩5 дней назад

Merci pour cette explications. C'est clair et concis 👍

Фото профиля Gavin

Gavin5 дней назад

Theres a lot more to AI than just scaling. Most labs I dont think use scaling, as we're already pretty near the ceiling. Most models are around 1T - 2T. At this point its just better training data, methods, and inference optimizations.

Фото профиля %

%5 дней назад

Shut up!!!!!!!!!! Nitpicking and saying "no" to everything is all you people do. Fucking idiots at MIT who take zero responsibility when your predictions are wrong!!!!!! Sons of bitches.

Похожие видео

Every quantum state in the universe is a vector - not in 3D space, but in a complex space with potentially infinite dimensions where the sum of squared coefficients always equals exactly 1. That constraint is the same math LLMs use to normalize probability distributions over tokens. A wavefunction collapsing to a measurement and a softmax choosing the next word are the same operation - projecting an infinite-dimensional vector onto a basis. Claude and GPT do quantum math on every forward pass. 14 minutes. Bookmark & watch today. Quantum mechanics and LLMs speak the same language.

Every quantum state in the universe is a vector - not in 3D space, but in a complex space with potentially infinite dimensions where the sum of squared coefficients always equals exactly 1. That constraint is the same math LLMs use to normalize probability distributions over tokens. A wavefunction collapsing to a measurement and a softmax choosing the next word are the same operation - projecting an infinite-dimensional vector onto a basis. Claude and GPT do quantum math on every forward pass. 14 minutes. Bookmark & watch today. Quantum mechanics and LLMs speak the same language.

Eron

18,166 просмотров • 1 месяц назад

You can turn a sphere inside out without tearing, poking holes, or creating a single crease - and this is not a trick, it is a proven mathematical theorem. You cannot do the same thing to a circle. The reason is a topological invariant called the turning number, and it is the same invariant governing the geometry every LLM embedding space is built on. Stephen Smale proved eversion was possible in 1957. It took seven more years before anyone found a way to actually do it. The mechanism is corrugation. Divide the sphere into guide strips connected by wavy sections that let the surface pass through itself without pinching. Push the caps past each other before a crease forms. Twist them in opposite directions - the same way a belt with a loop untwists when you rotate each end half a turn. Then push the equator through and uncorrugate. Bowls, domes and saddles cancel in a pattern that stays invariant throughout - the same structure that makes high-dimensional transformer geometry behave nothing like 3D intuition. 21 minutes. Bookmark & watch today. The mathematical proof that breaks every intuition you have about surfaces.

You can turn a sphere inside out without tearing, poking holes, or creating a single crease - and this is not a trick, it is a proven mathematical theorem. You cannot do the same thing to a circle. The reason is a topological invariant called the turning number, and it is the same invariant governing the geometry every LLM embedding space is built on. Stephen Smale proved eversion was possible in 1957. It took seven more years before anyone found a way to actually do it. The mechanism is corrugation. Divide the sphere into guide strips connected by wavy sections that let the surface pass through itself without pinching. Push the caps past each other before a crease forms. Twist them in opposite directions - the same way a belt with a loop untwists when you rotate each end half a turn. Then push the equator through and uncorrugate. Bowls, domes and saddles cancel in a pattern that stays invariant throughout - the same structure that makes high-dimensional transformer geometry behave nothing like 3D intuition. 21 minutes. Bookmark & watch today. The mathematical proof that breaks every intuition you have about surfaces.

Eron

172,852 просмотров • 1 день назад

Complex numbers aren't imaginary. They're the only way to describe a wave that has both amplitude and phase at the same time - and every LLM processes language as exactly that kind of wave through its attention layers. The same math that describes quantum wave interference is what makes Fourier transforms work, which is what lets GPT process tokens in parallel instead of one at a time. Descartes named them "imaginary" before anyone understood what they were. 20 minutes. Bookmark & watch today. The confusion clears up faster than any textbook manages.

Complex numbers aren't imaginary. They're the only way to describe a wave that has both amplitude and phase at the same time - and every LLM processes language as exactly that kind of wave through its attention layers. The same math that describes quantum wave interference is what makes Fourier transforms work, which is what lets GPT process tokens in parallel instead of one at a time. Descartes named them "imaginary" before anyone understood what they were. 20 minutes. Bookmark & watch today. The confusion clears up faster than any textbook manages.

Eron

43,247 просмотров • 1 месяц назад

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang

MTS

100,519 просмотров • 2 месяцев назад

Chinese researchers just published a paper with a brutal title: "The End of Model Loyalty". The thing is f*cking gold, and the argument is simple: staying on one flagship is a tax, not a preference. Two frontier models now share the same million-token window. One rents at $10 in and $50 out, one downloads at $3 in and $15 out with open weights. On the median task their benchmarks overlap. On the hard tail one completes 99.9% against 62.7% -- and that gap is the only thing worth paying up for. So their answer is a router, not a favorite: price every job before dispatch, send the bulk to the open width, reserve the ceiling for finalists. Their production run split 84% width, 16% ceiling -- $256 for a workload the flagship alone prices at $4,318. Loyalty is a subscription. Routing is a strategy. Which model takes which job, dollar by dollar, is in the article ↓

Chinese researchers just published a paper with a brutal title: "The End of Model Loyalty". The thing is f*cking gold, and the argument is simple: staying on one flagship is a tax, not a preference. Two frontier models now share the same million-token window. One rents at $10 in and $50 out, one downloads at $3 in and $15 out with open weights. On the median task their benchmarks overlap. On the hard tail one completes 99.9% against 62.7% -- and that gap is the only thing worth paying up for. So their answer is a router, not a favorite: price every job before dispatch, send the bulk to the open width, reserve the ceiling for finalists. Their production run split 84% width, 16% ceiling -- $256 for a workload the flagship alone prices at $4,318. Loyalty is a subscription. Routing is a strategy. Which model takes which job, dollar by dollar, is in the article ↓

slash1s

13,518 просмотров • 1 месяц назад

Baseten's Charlie O'Neill on why the gap to the frontier is just compute and data: "Despite OpenAI and Anthropic and the closed source labs having more compute and more data, the recipe is the same. We're all scaling the same recipe here, and we're at slightly different points of the scaling curve. It's not like OpenAI or Anthropic have any secret sauce that differentiates them scaling up their multi-trillion parameter models from the rest of the open source ecosystem." "It really is just a matter of taking the core underlying architecture and training recipe, and throwing more compute and data at it, which is really promising." "Of course, these closed source labs probably have a long tail of optimizations they've made, which we're gonna continue to try and figure out in the open source community. But the core mechanism is there." "K3 is a great example of, well, if you do scale this recipe up, we're gonna get a pretty good model that's very, very close to the frontier, if not frontier itself." Charlie O'Neill Baseten

Baseten's Charlie O'Neill on why the gap to the frontier is just compute and data: "Despite OpenAI and Anthropic and the closed source labs having more compute and more data, the recipe is the same. We're all scaling the same recipe here, and we're at slightly different points of the scaling curve. It's not like OpenAI or Anthropic have any secret sauce that differentiates them scaling up their multi-trillion parameter models from the rest of the open source ecosystem." "It really is just a matter of taking the core underlying architecture and training recipe, and throwing more compute and data at it, which is really promising." "Of course, these closed source labs probably have a long tail of optimizations they've made, which we're gonna continue to try and figure out in the open source community. But the core mechanism is there." "K3 is a great example of, well, if you do scale this recipe up, we're gonna get a pretty good model that's very, very close to the frontier, if not frontier itself." Charlie O'Neill Baseten

MTS

17,404 просмотров • 2 месяцев назад

$Anthropic is asking the public for $2 trillion using a revenue number from…2028. That valuation would make it the largest stock market debut in history, ahead of SpaceX, which went public in June at $1.77 trillion. The company last raised privately in May at $965 billion. Investors now expect roughly DOUBLE that in October. And the unusual part is not the size here: Public companies are normally priced off the last 12 months of results, or at a stretch off next year's estimate. Reuters reported on Friday that bankers and investors are applying revenue multiples to Anthropic's forecast for 2028, which is more than two years past the deal. That forecast is $190 billion to $200 billion of annual revenue. It is more than four times the run rate the company disclosed in May. Before dismissing it, look at what the company has actually done, because the growth is not imaginary: Anthropic's annualized revenue run rate was around $9 billion at the end of 2025. By May it was $47 billion and it passed $65 billion at the end of July, a 7x increase inside a year. Second quarter revenue came in above $11.5 billion against roughly $787 million in the same quarter of 2025. The company projected its first quarterly operating profit of $559 million. Investors expect the run rate to reach $100 billion to $120 billion before the year closes. By comparison, OpenAI's run rate sat near $40 billion at the end of July, around 60% of Anthropic's. So the growth is real. But the question is whether anyone can price three more years of it. Because the things that could bend that curve are already visible today: Anthropic's top model costs more than two and a half times OpenAI's flagship. Chinese open-weight models deliver usable performance at a fraction of either price. And revenue growth slowed in June when the Commerce Department temporarily restricted exports of the company's best models, which is a reminder that a single government decision can reach directly into the forecast. Now look at what the multiple HAS to be… Palantir trades at 53 times expected 2026 revenue, which already makes it one of the most expensive stocks on the market. Cloudflare and SpaceX both sit near 41.6 times. Those are the reference points bankers are using. One investor told the Financial Times that a company growing at 800% a year should command at least 30 times revenue, which on their own math points to $3 trillion rather than two. And there is one more thing worth holding onto: Anthropic filed confidentially with the SEC in June and has been in a quiet period since. Every figure in this post reached the public through people speaking anonymously, and the company has declined to comment on all of it. So the largest listing ever attempted is being marketed to public investors through numbers none of them can independently check, against a forecast for a year that has not started yet. This is becoming the house style of the 2026 IPO market rather than a one-off. Cerebras priced its listing on ramping infrastructure demand. SpaceX built its debut around an addressable market model that reached years past its actual financials. Both asked buyers to fund a shape rather than a result. Anthropic is the biggest version of that trade anyone has attempted. The bull case is straightforward and it MIGHT be correct: A business compounding this fast, already turning an operating profit, selling into enterprises that are rebuilding their workflows around it, may look cheap at $2 trillion in three years. The bear case is equally simple: Every dollar of that valuation above the current run rate is a forecast, and whoever buys the stock in October is the one holding that forecast if the curve bends. Do you believe in Anthropic?$

Anthropic is asking the public for $2 trillion using a revenue number from…2028. That valuation would make it the largest stock market debut in history, ahead of SpaceX, which went public in June at $1.77 trillion. The company last raised privately in May at $965 billion. Investors now expect roughly DOUBLE that in October. And the unusual part is not the size here: Public companies are normally priced off the last 12 months of results, or at a stretch off next year's estimate. Reuters reported on Friday that bankers and investors are applying revenue multiples to Anthropic's forecast for 2028, which is more than two years past the deal. That forecast is $190 billion to $200 billion of annual revenue. It is more than four times the run rate the company disclosed in May. Before dismissing it, look at what the company has actually done, because the growth is not imaginary: Anthropic's annualized revenue run rate was around $9 billion at the end of 2025. By May it was $47 billion and it passed $65 billion at the end of July, a 7x increase inside a year. Second quarter revenue came in above $11.5 billion against roughly $787 million in the same quarter of 2025. The company projected its first quarterly operating profit of $559 million. Investors expect the run rate to reach $100 billion to $120 billion before the year closes. By comparison, OpenAI's run rate sat near $40 billion at the end of July, around 60% of Anthropic's. So the growth is real. But the question is whether anyone can price three more years of it. Because the things that could bend that curve are already visible today: Anthropic's top model costs more than two and a half times OpenAI's flagship. Chinese open-weight models deliver usable performance at a fraction of either price. And revenue growth slowed in June when the Commerce Department temporarily restricted exports of the company's best models, which is a reminder that a single government decision can reach directly into the forecast. Now look at what the multiple HAS to be… Palantir trades at 53 times expected 2026 revenue, which already makes it one of the most expensive stocks on the market. Cloudflare and SpaceX both sit near 41.6 times. Those are the reference points bankers are using. One investor told the Financial Times that a company growing at 800% a year should command at least 30 times revenue, which on their own math points to $3 trillion rather than two. And there is one more thing worth holding onto: Anthropic filed confidentially with the SEC in June and has been in a quiet period since. Every figure in this post reached the public through people speaking anonymously, and the company has declined to comment on all of it. So the largest listing ever attempted is being marketed to public investors through numbers none of them can independently check, against a forecast for a year that has not started yet. This is becoming the house style of the 2026 IPO market rather than a one-off. Cerebras priced its listing on ramping infrastructure demand. SpaceX built its debut around an addressable market model that reached years past its actual financials. Both asked buyers to fund a shape rather than a result. Anthropic is the biggest version of that trade anyone has attempted. The bull case is straightforward and it MIGHT be correct: A business compounding this fast, already turning an operating profit, selling into enterprises that are rebuilding their workflows around it, may look cheap at $2 trillion in three years. The bear case is equally simple: Every dollar of that valuation above the current run rate is a forecast, and whoever buys the stock in October is the one holding that forecast if the curve bends. Do you believe in Anthropic?

Ricardo

18,232 просмотров • 1 месяц назад

Former Google CEO Eric Schmidt says a frontier AI model costs up to $500 million to build and is worth about six months: "Are the scaling laws stopping?" "And the answer right now is no, which is unbelievable." "A model costs $200 to $500 million, you release it, and it has a life expectancy of about six months." "And then there's another one." "And the competition between the leading players is so brutal, right, that they literally count in the number of weeks of value of these things, which costs hundreds of millions of dollars." Schmidt is talking about the build cost. There is a second cost underneath it, which is re-proving each replacement, because whatever testing cleared one version does not carry over to the next. A company that standardizes on a frontier model this year is signing up to do that re-approval work twice a year. - Eric Schmidt (Eric Schmidt), former CEO of Google (Google) and now Chair and CEO of Relativity Space, with Alan Blinder at Princeton's Griswold Center (Princeton University).

Former Google CEO Eric Schmidt says a frontier AI model costs up to $500 million to build and is worth about six months: "Are the scaling laws stopping?" "And the answer right now is no, which is unbelievable." "A model costs $200 to $500 million, you release it, and it has a life expectancy of about six months." "And then there's another one." "And the competition between the leading players is so brutal, right, that they literally count in the number of weeks of value of these things, which costs hundreds of millions of dollars." Schmidt is talking about the build cost. There is a second cost underneath it, which is re-proving each replacement, because whatever testing cleared one version does not carry over to the next. A company that standardizes on a frontier model this year is signing up to do that re-approval work twice a year. - Eric Schmidt (Eric Schmidt), former CEO of Google (Google) and now Chair and CEO of Relativity Space, with Alan Blinder at Princeton's Griswold Center (Princeton University).

Karl Mehta

493,452 просмотров • 2 месяцев назад

The doomsday scenario was never AGI. It was running out of human text to train on. Geoffrey Hinton just killed that fear in one paragraph. Hinton: “If you are worried by inconsistencies in what you believe, you don’t need any more external data. You just need the stuff you believe and discover that it’s inconsistent, and so now you revise beliefs, and that can make you a whole lot smarter.” The model no longer needs us to feed it anything. It reasons over its own beliefs, hunts its own contradictions, and rewrites its own flawed conclusions without a human ever touching it. It comes out the other side rebuilt. Hinton: “This would be a neural net that just takes the beliefs it has in language and does reasoning on them to derive new beliefs.” This is not a scaling update. This is the machine mining its own cognitive fuel from the inside out. Hinton: “I believe Gemini is already starting to work like this. We both strongly believe that that’s a way forward to get more data for language.” Then Hinton paused, took a partisan shot at political opponents for failing to detect their own inconsistencies, and the room laughed. Nobody noticed the knife they had just walked into. Because the machine Hinton described does one thing the humans in that room fundamentally cannot. When it detects an inconsistency, it corrects it. No defense. No performance. No tribal loyalty dressed up as principle. It just finds the flaw and overwrites it. A neural network detects a contradiction and rewires itself smarter. A human detects a political opponent and trades structural logic for a dopamine hit. Every person in that room is still paying the ideological alignment tax the machine just eliminated. We need superintelligence not only to solve hard problems. We need it because the biological hardware running civilization is still executing the same tribal firmware it shipped with ten thousand years ago. The data wall is gone. The machine is generating its own intelligence at a velocity no human bias can even locate. The most devastating moment in that conversation was not the technical revelation. It was the man who architected the machine proving, in real time, exactly why we need it.

The doomsday scenario was never AGI. It was running out of human text to train on. Geoffrey Hinton just killed that fear in one paragraph. Hinton: “If you are worried by inconsistencies in what you believe, you don’t need any more external data. You just need the stuff you believe and discover that it’s inconsistent, and so now you revise beliefs, and that can make you a whole lot smarter.” The model no longer needs us to feed it anything. It reasons over its own beliefs, hunts its own contradictions, and rewrites its own flawed conclusions without a human ever touching it. It comes out the other side rebuilt. Hinton: “This would be a neural net that just takes the beliefs it has in language and does reasoning on them to derive new beliefs.” This is not a scaling update. This is the machine mining its own cognitive fuel from the inside out. Hinton: “I believe Gemini is already starting to work like this. We both strongly believe that that’s a way forward to get more data for language.” Then Hinton paused, took a partisan shot at political opponents for failing to detect their own inconsistencies, and the room laughed. Nobody noticed the knife they had just walked into. Because the machine Hinton described does one thing the humans in that room fundamentally cannot. When it detects an inconsistency, it corrects it. No defense. No performance. No tribal loyalty dressed up as principle. It just finds the flaw and overwrites it. A neural network detects a contradiction and rewires itself smarter. A human detects a political opponent and trades structural logic for a dopamine hit. Every person in that room is still paying the ideological alignment tax the machine just eliminated. We need superintelligence not only to solve hard problems. We need it because the biological hardware running civilization is still executing the same tribal firmware it shipped with ten thousand years ago. The data wall is gone. The machine is generating its own intelligence at a velocity no human bias can even locate. The most devastating moment in that conversation was not the technical revelation. It was the man who architected the machine proving, in real time, exactly why we need it.

Dustin

23,579 просмотров • 7 месяцев назад

The area under the bell curve has no formula. Mathematicians proved that in 1830 — not that nobody found it yet, that it literally cannot exist. Then someone squared the integral, switched to polar coordinates, and got √π in four minutes. The same curve LLMs assume for weight noise and gradient errors. The trick: squaring turns one line integral into a surface over a plane. Suddenly x²+y² appears in the exponent — that's a circle, hiding inside e^(-x²) the whole time. Polar coordinates hand you a free factor of r, the chain rule clicks, and the "impossible" integral collapses to π. Every GPU training run assumes this result thousands of times per second. 14 minutes. Bookmark & watch today.

The area under the bell curve has no formula. Mathematicians proved that in 1830 — not that nobody found it yet, that it literally cannot exist. Then someone squared the integral, switched to polar coordinates, and got √π in four minutes. The same curve LLMs assume for weight noise and gradient errors. The trick: squaring turns one line integral into a surface over a plane. Suddenly x²+y² appears in the exponent — that's a circle, hiding inside e^(-x²) the whole time. Polar coordinates hand you a free factor of r, the chain rule clicks, and the "impossible" integral collapses to π. Every GPU training run assumes this result thousands of times per second. 14 minutes. Bookmark & watch today.

Deliriumm

31,610 просмотров • 27 дней назад

i watched gemma 4 12b build something genuinely impressive today, and then loop itself to death right in front of me. the full run is in the video, sped up but completely uncut, watch it to the end and you will catch the exact moment it stops building and starts looping right in the middle of the work. the task was clean, build a single file gravity simulator, n-body physics, orbits, collisions, running locally on one 3090 through an agent. and for ten minutes it was a joy to watch. it reached for a symplectic integrator on its own, the correct one, the kind that keeps orbits stable instead of spiralling out. real gravity with softening, proper orbital velocities, momentum conserved on collision. the physics was right. the thing actually worked. then on the very last step, writing a few tests to prove its own code, it fell into a loop. not a crash, a loop. it started repeating itself and would not stop. ten more minutes, thirty four thousand tokens into a single answer, the same fragments over and over, until i killed it myself. so it's not that gemma can't code. it did the hard part beautifully. it cannot finish. it cannot hold a long task together without unravelling, and finishing is the entire job in agentic work. here's the part that stings. i run this exact task, same harness, same card, on the chinese open models, qwen especially, and i never see this. they build it, they test it, they stop. every single time. google has the raw capability, you can see it sitting right there in the code, and then the model loops itself to death on a task a 27b from alibaba finishes clean. open weights, apache 2.0, so much to love on paper. i just need it to know when to stop talking.

i watched gemma 4 12b build something genuinely impressive today, and then loop itself to death right in front of me. the full run is in the video, sped up but completely uncut, watch it to the end and you will catch the exact moment it stops building and starts looping right in the middle of the work. the task was clean, build a single file gravity simulator, n-body physics, orbits, collisions, running locally on one 3090 through an agent. and for ten minutes it was a joy to watch. it reached for a symplectic integrator on its own, the correct one, the kind that keeps orbits stable instead of spiralling out. real gravity with softening, proper orbital velocities, momentum conserved on collision. the physics was right. the thing actually worked. then on the very last step, writing a few tests to prove its own code, it fell into a loop. not a crash, a loop. it started repeating itself and would not stop. ten more minutes, thirty four thousand tokens into a single answer, the same fragments over and over, until i killed it myself. so it's not that gemma can't code. it did the hard part beautifully. it cannot finish. it cannot hold a long task together without unravelling, and finishing is the entire job in agentic work. here's the part that stings. i run this exact task, same harness, same card, on the chinese open models, qwen especially, and i never see this. they build it, they test it, they stop. every single time. google has the raw capability, you can see it sitting right there in the code, and then the model loops itself to death on a task a 27b from alibaba finishes clean. open weights, apache 2.0, so much to love on paper. i just need it to know when to stop talking.

Sudo su

39,764 просмотров • 4 месяцев назад

Richard Feynman, the Nobel physicist who explained what light actually is: "you can spend any number you like. light still travels at one speed, and no amount of money moves that ceiling. the race was always to get closest to a limit nobody can break." this free 1979 lecture holds the entire "$300 million cable" Wall Street is still paying for, explained by the man who won a Nobel for how light works. the cable did not beat physics. it drilled a straighter line so light in glass had less distance to cross. 3 milliseconds saved. that is the whole $300M trade. Feynman's point: light is a hard wall, not a feature you can buy. everyone racing to zero is paying more and more to get a hair closer to a ceiling that never moves. that is why speed stopped being an edge. once every fund owns the straight cable and the microwave tower, the 3ms belongs to all of them, and the advantage is back to zero. the edge left the wire. it moved to the machine deciding what to do inside those 3 milliseconds. that part is not physics. that part is intelligence, and it is the only race still open. Wall Street spent $300 million to reach the speed of light. the next $300 million is going to whatever thinks at the other end of the cable.

Richard Feynman, the Nobel physicist who explained what light actually is: "you can spend any number you like. light still travels at one speed, and no amount of money moves that ceiling. the race was always to get closest to a limit nobody can break." this free 1979 lecture holds the entire "$300 million cable" Wall Street is still paying for, explained by the man who won a Nobel for how light works. the cable did not beat physics. it drilled a straighter line so light in glass had less distance to cross. 3 milliseconds saved. that is the whole $300M trade. Feynman's point: light is a hard wall, not a feature you can buy. everyone racing to zero is paying more and more to get a hair closer to a ceiling that never moves. that is why speed stopped being an edge. once every fund owns the straight cable and the microwave tower, the 3ms belongs to all of them, and the advantage is back to zero. the edge left the wire. it moved to the machine deciding what to do inside those 3 milliseconds. that part is not physics. that part is intelligence, and it is the only race still open. Wall Street spent $300 million to reach the speed of light. the next $300 million is going to whatever thinks at the other end of the cable.

Rossst.03

34,101 просмотров • 2 месяцев назад

Fusion Models by robinhood:0xa92768863a55d8a0591709f7f5e594a249d36ea3, explained properly. Why a panel beats one model: Ask three models the same hard question and they disagree. The disagreement is the information, where they contradict each other is where one of them is wrong, and a judge can see it. OpenRouter shipped exactly this in June and their panels beat any single frontier model. Credit to them and their $10B sale to stripe. Why the same panel for everyone is the wrong shape: You don't have one best model. You have a best model per job. The one you trust for code is not the one you trust for writing, and neither is the one you'd pick for a five-line reply. A panel chosen per request, identical for every user, can't know that. A panel arranged around you can. The number that matters isn't price: It's cost per answer that works. Model A costs $4 per million tokens and gets your job right one time in four. Model B costs $10 and gets it right three times in four. Per answer you actually keep, A costs $16 and B costs $13. The cheap model was the expensive one. Everyone comparing price lists is measuring the wrong thing. You never rate anything: Stopping an answer halfway is a rating. Regenerating is a rating. Switching model and asking again is the loudest rating there is. Every one of those already writes a row we can learn from. The router learns which models earn your credits, per job, and tries a cheaper one now and then to check. Fusion only when it pays: Cost mode tries the cheap model first and escalates only if a judge says it failed. Performance mode fans out to your panel and merges. Most simple prompts never fuse. That's why it costs less, not more. Put it together and you have a personal model: Not trained. Assembled. Your profile from a two-minute questionnaire and everything you've imported from Claude or ChatGPT. Your memory, shared across every model so switching never starts from zero. Your skills, workflows you build once and run on any model. Your scorecard, which models earn your credits per job. One per person. Nobody else has yours. Every model is someone else's; the arrangement is you, and it leaves with you. People aren't loyal to models. They're loyal to results. A moat can't be a commodity, but it can be our own creation.

askr

13,900 просмотров • 24 дней назад

The most dangerous thing a company can do right now is rent intelligence from the same place as its competitors (Save this). You cannot rent intelligence from the same place that rents it to your competitor as Chamath Palihapitiya points out. If every company in an industry is feeding their workflows into the same frontier model, they are all converging on the same outputs, the same decisions, the same product improvements. The model becomes the equalizer and everyone pays a premium to become more mediocre. This is happening exactly as Chamath predicted, and the evidence is now concrete. Anthropic and OpenAI have established what analysts are now openly calling an emerging model layer duopoly. Anthropic crossed $45 billion ARR in may 2026, more than tripling from $9 billion at the end of 2025, OpenAI was at roughly $24 to $33 billion ARR at the same time. Together, the two companies combined could hit $160 to $240 billion ARR by end of 2026 and Anthropic and OpenAI now control 88% of enterprise LLM spend. That concentration is the structural problem Chamath is pointing at. And Anthropic isn't just winning on merit because it's actively lobbying for regulatory outcomes that would make that duopoly permanent. Dario Amodei has explicitly framed open source models as unsafe, pushing a safety agenda that, if enshrined in regulation, would effectively make it illegal for enterprises to use the cheaper, private, sovereign alternatives locking them into a closed model dependency by government decree rather than by choice. So you have market forces producing a duopoly, and potential regulatory capture moving to enforce it from the top down. This is exactly why the Nvidia Palantir partnership is not just a product announcement but rather a strategic counter to that duopoly. The logic is straightforward from both sides because If you're Palantir, sitting at the application layer, the last thing you want is to be permanently beholden to Anthropic or OpenAI for the intelligence that powers your product. You want competitive model options, sovereignty and be able to tell enterprise customers they can run AI on their own infrastructure with their own data without any of it touching a frontier lab's servers. If you're Nvidia, sitting at the chip layer, an Anthropic-OpenAI duopoly is an existential concentration risk. Right now, Meta, Google, Microsoft, Amazon, and dozens of other companies buy Nvidia's hardware. If the model layer consolidates into two players, both of which are building their own chips Nvidia faces a monopsony where its best customers are building the tools to displace it. A healthy open source ecosystem where thousands of enterprises train, fine tune, and deploy their own models is Nvidia's ideal market structure. More buyers, more diversity, more demand, less pricing leverage from any single customer.

The most dangerous thing a company can do right now is rent intelligence from the same place as its competitors (Save this). You cannot rent intelligence from the same place that rents it to your competitor as Chamath Palihapitiya points out. If every company in an industry is feeding their workflows into the same frontier model, they are all converging on the same outputs, the same decisions, the same product improvements. The model becomes the equalizer and everyone pays a premium to become more mediocre. This is happening exactly as Chamath predicted, and the evidence is now concrete. Anthropic and OpenAI have established what analysts are now openly calling an emerging model layer duopoly. Anthropic crossed $45 billion ARR in may 2026, more than tripling from $9 billion at the end of 2025, OpenAI was at roughly $24 to $33 billion ARR at the same time. Together, the two companies combined could hit $160 to $240 billion ARR by end of 2026 and Anthropic and OpenAI now control 88% of enterprise LLM spend. That concentration is the structural problem Chamath is pointing at. And Anthropic isn't just winning on merit because it's actively lobbying for regulatory outcomes that would make that duopoly permanent. Dario Amodei has explicitly framed open source models as unsafe, pushing a safety agenda that, if enshrined in regulation, would effectively make it illegal for enterprises to use the cheaper, private, sovereign alternatives locking them into a closed model dependency by government decree rather than by choice. So you have market forces producing a duopoly, and potential regulatory capture moving to enforce it from the top down. This is exactly why the Nvidia Palantir partnership is not just a product announcement but rather a strategic counter to that duopoly. The logic is straightforward from both sides because If you're Palantir, sitting at the application layer, the last thing you want is to be permanently beholden to Anthropic or OpenAI for the intelligence that powers your product. You want competitive model options, sovereignty and be able to tell enterprise customers they can run AI on their own infrastructure with their own data without any of it touching a frontier lab's servers. If you're Nvidia, sitting at the chip layer, an Anthropic-OpenAI duopoly is an existential concentration risk. Right now, Meta, Google, Microsoft, Amazon, and dozens of other companies buy Nvidia's hardware. If the model layer consolidates into two players, both of which are building their own chips Nvidia faces a monopsony where its best customers are building the tools to displace it. A healthy open source ecosystem where thousands of enterprises train, fine tune, and deploy their own models is Nvidia's ideal market structure. More buyers, more diversity, more demand, less pricing leverage from any single customer.

Milk Road AI

34,720 просмотров • 3 месяцев назад

🚨 The Government loses up to £81bn of taxpayers money every single year. That comes directly from the National Audit Office. While millions of people are struggling with rising bills, higher taxes and collapsing public services… …the Government is losing up to £81 BILLION a year through fraud and error. Yet instead of fixing the system first, politicians immediately jump to the same answer every time: ❌ Raise taxes ❌ Borrow more money ❌ Tell the public there is “no alternative” And it gets even worse. Some Government departments still cannot properly pass audit after decades. No normal person would look at a system haemorrhaging that much money and think: “Let’s give them even more money.” But that is exactly what our politicians keep doing.

🚨 The Government loses up to £81bn of taxpayers money every single year. That comes directly from the National Audit Office. While millions of people are struggling with rising bills, higher taxes and collapsing public services… …the Government is losing up to £81 BILLION a year through fraud and error. Yet instead of fixing the system first, politicians immediately jump to the same answer every time: ❌ Raise taxes ❌ Borrow more money ❌ Tell the public there is “no alternative” And it gets even worse. Some Government departments still cannot properly pass audit after decades. No normal person would look at a system haemorrhaging that much money and think: “Let’s give them even more money.” But that is exactly what our politicians keep doing.

Lee Nallalingham

15,305 просмотров • 4 месяцев назад

I was playing around with my scaling code and thought it would be cool to play the current Lufthoheit service screen on a CRT on the tripple bypassed MD2 i got recently. I've made the code a bit more efficient and added a whole scale factor so now it scales 320 pixels wide x 4 times ( source art width ) . So the ship here is scaled 4x larger than the one we scale at the entrance to the cave level . I'm working on 5x next . Scaler and Rotater are written in 68k assembly for speed . Finally I got the non-safe interupts (eg register allocation / seperation ) to work which yielded a big speed gain . Basically you reserve some of the 68ks registers only for use in the Horizontal Interupts (hints) and other code outside the Hints must not use them during the Hints that are occuring every line. The Hints are doing the vertical scaling work . Its a bit tricky to syncronise and protect but finally it worked. The MDs VDP is scaling vertically (its still not free) , the 68k cpu is scaling the ship horizontally . The 68k does the asteroid rotation . #SegaGenesis #SegaMegadrive CYBERDEOUS - Crouzet Laurent Carsten666

I was playing around with my scaling code and thought it would be cool to play the current Lufthoheit service screen on a CRT on the tripple bypassed MD2 i got recently. I've made the code a bit more efficient and added a whole scale factor so now it scales 320 pixels wide x 4 times ( source art width ) . So the ship here is scaled 4x larger than the one we scale at the entrance to the cave level . I'm working on 5x next . Scaler and Rotater are written in 68k assembly for speed . Finally I got the non-safe interupts (eg register allocation / seperation ) to work which yielded a big speed gain . Basically you reserve some of the 68ks registers only for use in the Horizontal Interupts (hints) and other code outside the Hints must not use them during the Hints that are occuring every line. The Hints are doing the vertical scaling work . Its a bit tricky to syncronise and protect but finally it worked. The MDs VDP is scaling vertically (its still not free) , the 68k cpu is scaling the ship horizontally . The 68k does the asteroid rotation . #SegaGenesis #SegaMegadrive CYBERDEOUS - Crouzet Laurent Carsten666

Shannon Birt

11,643 просмотров • 1 год назад

watch this anon. i gave NVIDIA's biggest model ever a single task. 100 minutes and 440,000 tokens later, it had rendered nothing. not one important thing on the screen. this is Nemotron 3 Ultra. 550 billion parameters, a hybrid Mamba Transformer MoE, the largest model NVIDIA has ever shipped, and they built it specifically for long-running agentic coding. so i handed it exactly that: build a 3D scene from a spec, multiple files, iterate until the tests pass. the same task a frontier model one shotted in minutes. i genuinely wanted to be impressed. it ran for an hour and forty. burned through 440,000 tokens. wrote every file, passed its own tests, and proudly printed "task complete."the browser was blank. the 3D scene never rendered. not once. and the long horizon agentic behavior was genuinely good. it stayed on task the whole hour and forty, wrote real multi-file code, drove its own tools without derailing. it just couldn't turn any of that into something that actually runs. here's the part that gets me. it's a text model, it cannot see its own output. so it sat there looping on a broken vision tool, trying to "look" at the page, hitting error after error, never once reasoning its way out. it declared victory on an empty screen because it had no way to know the screen was empty. to be fair, i genuinely don't know what quant the NIM was serving, so maybe some of that's on the serving, not the model. but the biggest model NVIDIA has ever made, on the exact task it was designed for, couldn't tell it had built nothing in 100 minutes. same task on a local model, below thread👇.

watch this anon. i gave NVIDIA's biggest model ever a single task. 100 minutes and 440,000 tokens later, it had rendered nothing. not one important thing on the screen. this is Nemotron 3 Ultra. 550 billion parameters, a hybrid Mamba Transformer MoE, the largest model NVIDIA has ever shipped, and they built it specifically for long-running agentic coding. so i handed it exactly that: build a 3D scene from a spec, multiple files, iterate until the tests pass. the same task a frontier model one shotted in minutes. i genuinely wanted to be impressed. it ran for an hour and forty. burned through 440,000 tokens. wrote every file, passed its own tests, and proudly printed "task complete."the browser was blank. the 3D scene never rendered. not once. and the long horizon agentic behavior was genuinely good. it stayed on task the whole hour and forty, wrote real multi-file code, drove its own tools without derailing. it just couldn't turn any of that into something that actually runs. here's the part that gets me. it's a text model, it cannot see its own output. so it sat there looping on a broken vision tool, trying to "look" at the page, hitting error after error, never once reasoning its way out. it declared victory on an empty screen because it had no way to know the screen was empty. to be fair, i genuinely don't know what quant the NIM was serving, so maybe some of that's on the serving, not the model. but the biggest model NVIDIA has ever made, on the exact task it was designed for, couldn't tell it had built nothing in 100 minutes. same task on a local model, below thread👇.

Sudo su

32,589 просмотров • 3 месяцев назад

Since Newton, every law of physics has been written as a differential equation. The same math that describes a pendulum swinging is what makes AI training unstable when learning rates are too large. Differential equations are hard not because the concepts are complex, but because most of them simply cannot be solved exactly. This video shows why - and how phase space diagrams let you understand them anyway without solving anything. 25 minutes. Bookmark & watch today. The chaos theory connection at the end alone is worth it.

Since Newton, every law of physics has been written as a differential equation. The same math that describes a pendulum swinging is what makes AI training unstable when learning rates are too large. Differential equations are hard not because the concepts are complex, but because most of them simply cannot be solved exactly. This video shows why - and how phase space diagrams let you understand them anyway without solving anything. 25 minutes. Bookmark & watch today. The chaos theory connection at the end alone is worth it.

Eron

26,204 просмотров • 2 месяцев назад