Loading video...

Video Failed to Load

Go Home

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry...

15,565 views • 1 month ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

NVIDIA CEO Jensen Huang says one scaling law multiplies AI faster than NVIDIA can hire engineers. Most people know three AI scaling laws. Pre-training. Post-training. Test-time. Each one multiplies intelligence by throwing more compute at a different stage. Jensen Huang says there's a fourth and it's the one that will dominate... Agentic scaling law. "During test time, that agentic system goes off and does research, bangs on databases, uses tools," Huang says. "And one of the most important things it does is spawn off a whole bunch of sub-agents." That's the multiplier. One AI worker can become a team. Then a department. Then a company. "It's so much easier to scale NVIDIA by hiring more employees than it is to scale myself," Huang says. Now imagine scaling without a payroll constraint. "The agentic scaling law — it's kind of like multiplying AI," Huang says. "We could spin off agents as fast as you want to spin off agents." Each agent spins off sub-agents. Each sub-agent spins off more. The compute requirement compounds inside a single query. And every agent generates new data, new experiences, new edge cases. "Wow, this is really good. We ought to memorize this," Huang says. "That data set comes back to pre-training." The four scaling laws don't compete. They feed each other. Agentic systems produce data, which feeds pre-training, which smartens the base model, which enables better agents, which produce more data. A flywheel that compounds forever. The companies pricing in three scaling laws are mispricing the fourth. The fourth eats the other three for lunch. P.S. Pull the thread on any story like this and you'll find the hidden incentive at the other end. As Munger said: "Show me the incentive and I'll show you the outcome." So I wrote a short book on how to spot them and design your own. Comment "INCENTIVES" and I'll send you the details. If you're new here, follow GeniusThinking for content on the greatest minds in economics, psychology, and history. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

GeniusThinking

92,806 views • 1 month ago

Dario Amodei just revealed that the AI training bottleneck everyone is worried about doesn’t exist anymore. The industry spent years obsessed with scraping the open web. More data. More text. More human output to feed the models. Amodei: “I don’t think data is quite the most central thing anymore.” The shift is fundamental. Amodei: “Static data is becoming less important. A lot of the data we use today is RL environments that we train on. Dynamic data that the model creates itself.” Not scraped. Not licensed. Not written by humans. Generated by the model through pure trial and error. When you train on complex math or agentic coding, you don’t feed it a textbook. You give it an environment. The model experiments. Fails. Adjusts. Tries again. Amodei: “You’re getting some math problems and the model experiments with trying the math problems.” It generates its own experience. Millions of iterations. Each one building on the last. No human required. This destroys the entire narrative around AI hitting a data wall. You cannot throttle a competitor by locking down copyright. Cannot slow the race by putting up a paywall. When a model learns through its own synthetic experience, the open web becomes irrelevant. The only true bottleneck left is compute. And this is where the geopolitical stakes become impossible to overstate. The nation that wins the compute race doesn’t just build smarter models. It builds models that generate their own intelligence, compounding on themselves, iterating past every limit human knowledge ever imposed. We are no longer training AI on the past. We are letting it simulate the future. The machine has stopped reading the dictionary. It’s doing the math itself now.

Dustin

149,610 views • 4 months ago

To everyone wondering if Tesla's FSD moat has eroded thanks to Nvidia's keynote - here's the answer: Think of Nvidia as a seller of a toolkit. You can buy pieces built specifically for a job but once you have the tools you still have to build the entire project. In this case...a model Many seem to be in awe over Nvidia's Cosmos which is a platform that includes World Foundation Models that can generate synthetic data for training AI systems like autonomous vehicles I'll go into much more depth in today's episode but here's a clip of Ashok Elluswamy at CVPR '23 explaining how Tesla is already using a similar approach to augment its real world data set This also doesn't even consider the fact that much of the auto industry using Nvidia "tools" will be forced to pay 50%+ margins just to buy the toolkit and will be locked in to Nvidia's system Nor does this touch on legacy auto needing to hire top ML engineering talent to actually put these tools to work The path to autonomy will be real world data as the foundation and simulations/synthetic data as a supplement. There is no path to autonomy with synthetic data alone. More to come later $TSLA As Elon said earlier this year, "it's remarkable how quickly we run out of human-created data. Reality itself and synthetic data ftw" "What you are seeing here is purely generated video sequences - given the past videos the network predicts some sample from the future, hopefully the most likely sample. It is being predicted not just for one camera, but it predicts for all 8 cameras around the car jointly" - Ashok

Dillon Loomis

60,753 views • 1 year ago

Coinbase CEO Explains “Reverse Prompting” and the Rise of the AI CEO Brian Armstrong: “One of the big pushes we made in the last year was we got our own internal hosted AI model that was connected to all of our data sources, right?” “So it's like every Slack message, every Google doc, Salesforce data, Confluence, you know.” “So now the data is all aggregated and I've started to ask it really… it's not just like prompting it, ‘Hey, can you write this kind of memo for me,’ or something.” “I'm asking these AI agents now, ‘As CEO, what should I be aware of in the company that I might not be aware of?’ And it'll tell me, ‘Did you know that there's actually disagreement on this team about the strategy?’ And I was like, actually, I didn't know that.” “This is like reverse prompting. So instead of telling the AI agent what you want it to do, you ask it what you should be thinking more about.” @jason: “It's a mentor. It's a coach.” Brian: “Yeah. Like, what could make me a better CEO? And it's like, ‘Well, I looked at how you spent your time in the last quarter and here's how you said that you wanted to spend it, but you actually spent 32% of your time on this instead of 20%.’” “I've asked it other questions like, ‘What's the thing that I changed my mind on the most over the last year?’ Things like that.” “It'll prompt you with information you should be thinking about instead of the other way around.” Thanks to our partner for making this happen!: Our episode is sponsored by the New York Stock Exchange - a modern marketplace and exchange for building the future. It all happens at the NYSE 🏛.

The All-In Podcast

80,524 views • 5 months ago