Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry...

15,565 Aufrufe • vor 3 Monaten •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."

High Signal AI

21,696 Aufrufe • vor 1 Monat

NVIDIA CEO Jensen Huang says one scaling law multiplies AI faster than NVIDIA can hire engineers. Most people know three AI scaling laws. Pre-training. Post-training. Test-time. Each one multiplies intelligence by throwing more compute at a different stage. Jensen Huang says there's a fourth and it's the one that will dominate... Agentic scaling law. "During test time, that agentic system goes off and does research, bangs on databases, uses tools," Huang says. "And one of the most important things it does is spawn off a whole bunch of sub-agents." That's the multiplier. One AI worker can become a team. Then a department. Then a company. "It's so much easier to scale NVIDIA by hiring more employees than it is to scale myself," Huang says. Now imagine scaling without a payroll constraint. "The agentic scaling law — it's kind of like multiplying AI," Huang says. "We could spin off agents as fast as you want to spin off agents." Each agent spins off sub-agents. Each sub-agent spins off more. The compute requirement compounds inside a single query. And every agent generates new data, new experiences, new edge cases. "Wow, this is really good. We ought to memorize this," Huang says. "That data set comes back to pre-training." The four scaling laws don't compete. They feed each other. Agentic systems produce data, which feeds pre-training, which smartens the base model, which enables better agents, which produce more data. A flywheel that compounds forever. The companies pricing in three scaling laws are mispricing the fourth. The fourth eats the other three for lunch. P.S. Pull the thread on any story like this and you'll find the hidden incentive at the other end. As Munger said: "Show me the incentive and I'll show you the outcome." So I wrote a short book on how to spot them and design your own. Comment "INCENTIVES" and I'll send you the details. If you're new here, follow GeniusThinking for content on the greatest minds in economics, psychology, and history. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

GeniusThinking

93,264 Aufrufe • vor 3 Monaten

Dario Amodei just revealed that the AI training bottleneck everyone is worried about doesn’t exist anymore. The industry spent years obsessed with scraping the open web. More data. More text. More human output to feed the models. Amodei: “I don’t think data is quite the most central thing anymore.” The shift is fundamental. Amodei: “Static data is becoming less important. A lot of the data we use today is RL environments that we train on. Dynamic data that the model creates itself.” Not scraped. Not licensed. Not written by humans. Generated by the model through pure trial and error. When you train on complex math or agentic coding, you don’t feed it a textbook. You give it an environment. The model experiments. Fails. Adjusts. Tries again. Amodei: “You’re getting some math problems and the model experiments with trying the math problems.” It generates its own experience. Millions of iterations. Each one building on the last. No human required. This destroys the entire narrative around AI hitting a data wall. You cannot throttle a competitor by locking down copyright. Cannot slow the race by putting up a paywall. When a model learns through its own synthetic experience, the open web becomes irrelevant. The only true bottleneck left is compute. And this is where the geopolitical stakes become impossible to overstate. The nation that wins the compute race doesn’t just build smarter models. It builds models that generate their own intelligence, compounding on themselves, iterating past every limit human knowledge ever imposed. We are no longer training AI on the past. We are letting it simulate the future. The machine has stopped reading the dictionary. It’s doing the math itself now.

Dustin

149,633 Aufrufe • vor 6 Monaten

Coinbase CEO Explains “Reverse Prompting” and the Rise of the AI CEO Brian Armstrong: “One of the big pushes we made in the last year was we got our own internal hosted AI model that was connected to all of our data sources, right?” “So it's like every Slack message, every Google doc, Salesforce data, Confluence, you know.” “So now the data is all aggregated and I've started to ask it really… it's not just like prompting it, ‘Hey, can you write this kind of memo for me,’ or something.” “I'm asking these AI agents now, ‘As CEO, what should I be aware of in the company that I might not be aware of?’ And it'll tell me, ‘Did you know that there's actually disagreement on this team about the strategy?’ And I was like, actually, I didn't know that.” “This is like reverse prompting. So instead of telling the AI agent what you want it to do, you ask it what you should be thinking more about.” @jason: “It's a mentor. It's a coach.” Brian: “Yeah. Like, what could make me a better CEO? And it's like, ‘Well, I looked at how you spent your time in the last quarter and here's how you said that you wanted to spend it, but you actually spent 32% of your time on this instead of 20%.’” “I've asked it other questions like, ‘What's the thing that I changed my mind on the most over the last year?’ Things like that.” “It'll prompt you with information you should be thinking about instead of the other way around.” Thanks to our partner for making this happen!: Our episode is sponsored by the New York Stock Exchange - a modern marketplace and exchange for building the future. It all happens at the NYSE 🏛.

The All-In Podcast

80,524 Aufrufe • vor 7 Monaten