High Signal AI
@HighSignal_AI • 67,214 subscribers
Signal in the AI noise
Videos
Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."
High Signal AI20,347 views • 2 days ago
Alex Hormozi's 5-step productivity system that he stole from Elon Musk:
High Signal AI325,059 views • 1 year ago
Eric Schmidt on what he calls "the San Francisco consensus": "There's a group of people that I work with. They're all in San Francisco and they've all basically convinced themselves that in the next two to four year, the average is three years the entire world will change." He's careful to flag the catch in that word "consensus": "It's true that it's a consensus, but it's not necessarily true that the consensus is true." So why do these people believe it? Eric walks through the reasoning. First came language-to-language models. "ChatGPT is the best one, they did a great job," he says, with everyone else catching up. Then came two additions: reasoning, and memory inside these systems. Put those together and you get what he calls the agentic revolution: "The agentic revolution… can be understood as language in, memory and language, language out." To make it concrete, he uses a deliberately mundane example. Say he wants to build a house in California. One agent finds the lot. Another works out the rules. Another designs the building. Another selects the contractor. And, at least in America, another agent sues the contractor when the house doesn't work. The point of such a "stupid example," as he puts it, is exactly that it's ordinary: "I just gave you a workflow example that's true of every business, every government and every group human activity." That's why he thinks the consensus carries weight: "When you understand that the agentic revolution and the reasoning revolution together really changed the way we operate as humans, then you understand why the San Francisco consensus is so powerful." On the reasoning side, he points to models that work a problem forward and backward: "It will blow your mind away, especially when you ask it a question that you have no idea what the question is about and you watch it reason." And he cites the numbers he's seeing: "Google now has a math model that is at the 90 percentile of math graduate students." More broadly, he claims, "we now have systems now that are 90% of all of the graduate school skills. You know, math, physics, and so forth." When something can do that, he argues, "it will happen at a big scale." This is where the consensus reaches its sharpest point: recursive self-improvement. "There's a moment when what is called recursive self-improvement, the system begins to learn on itself, where it goes forward at a rate that is impossible for us to understand. It becomes combinatorial in a way that we as humans do not understand." His verdict on all of it is two-sided: "This is both incredibly exciting and also very worrisome."
High Signal AI31,085 views • 1 month ago
Jensen Huang on why China is innovating so fast: NVIDIA CEO Jensen Huang explains what's driving China's pace of innovation and it starts with a counterintuitive move on technology itself. "There's no sense keeping technology hidden. You might as well put it on open source. And so the open source community then amplifies, accelerates the innovation process." The result, he says, is a compounding effect: open source attracts talent, talent drives rapid iteration, and fierce domestic competition between companies pushes the ceiling even higher. "So you get this rapid incredible great talent, rapid innovation because of open source and just the nature of friends. Insane competition among the companies. What emerges is incredible stuff." But Jensen argues the technology layer is only part of the story. The deeper edge is cultural. It was decades in the making: "This is the fastest innovating country in the world today. And this is something that has… everything that I've just said is fundamental to just how the kids were grown. The fact that they have excellent education, the fact that their parents want them to do well in school, the fact that their culture is that way." In other words: a generation raised to value education and rigor showed up at exactly the moment the technology curve went exponential. Lex Fridman adds the cultural detail that ties it together: "Plus, culturally, it's pretty cool to be an engineer." That framing leads Jensen to his sharpest contrast. One about who actually runs each country: "It's a builder nation. Our country's leaders, incredible, but they're mostly lawyers… rule of law, governing. Their country was built out of poverty. And so most of their leaders are incredible engineers, some of the brightest minds."
High Signal AI41,241 views • 2 months ago
"Courage is far shorter in supply than genius." - Peter Thiel
High Signal AI191,190 views • 1 year ago
Steve Jobs explains the difference between a great idea and a great product:
High Signal AI148,966 views • 1 year ago
