Loading video...

Video Failed to Load

Go Home

From a scrappy side project built to solve their own LLM optimization problems to becoming the industry’s de-facto independent scoreboard, Micah Hill-Smith and George Cameron went through the arc of launching Artificial Analysis for free, paying benchmarking costs out of pocket, and growing it into what many now call...

latentspacepod's profile picture

Latent.Space

25,535 subscribers

33,143 views • 5 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

We’re unveiling a new look for Artificial Analysis! We’ve come a long way since launching Artificial Analysis over 2 years ago. Today, we benchmark 400+ models, 50+ inference providers, and benchmark not only language models but also image, video, speech, music, hardware, and agents. Our mission to support the AI ecosystem with independent benchmarking remains the same, but our brand and website refresh is designed to better reflect how much we’ve grown and how much further we plan to go. A huge thank you to everyone who has been part of the Artificial Analysis community along the way: from developers choosing models and building agents, to labs, inference and hardware providers, and fellow independent researchers.
0:25

We’re unveiling a new look for Artificial Analysis! We’ve come a long way since launching Artificial Analysis over 2 years ago. Today, we benchmark 400+ models, 50+ inference providers, and benchmark not only language models but also image, video, speech, music, hardware, and agents. Our mission to support the AI ecosystem with independent benchmarking remains the same, but our brand and website refresh is designed to better reflect how much we’ve grown and how much further we plan to go. A huge thank you to everyone who has been part of the Artificial Analysis community along the way: from developers choosing models and building agents, to labs, inference and hardware providers, and fellow independent researchers.

Artificial Analysis

18,045 views • 2 months ago

Discussed the future of AI with Professor Zhang Weiwei: “We need to think ahead. We need to think about what we should do right now with artificial intelligence. When young children interact with their AI assistant, and it becomes closer to them than their parents, their friends, their nannies, preschool, or daycare. And this artificial intelligence keeps learning. And as its subject of study, as the textbooks for its own training, it uses Western platforms, which means it uses Western narratives. And very soon, we will have a completely different history of the world in their heads.”
0:27

Discussed the future of AI with Professor Zhang Weiwei: “We need to think ahead. We need to think about what we should do right now with artificial intelligence. When young children interact with their AI assistant, and it becomes closer to them than their parents, their friends, their nannies, preschool, or daycare. And this artificial intelligence keeps learning. And as its subject of study, as the textbooks for its own training, it uses Western platforms, which means it uses Western narratives. And very soon, we will have a completely different history of the world in their heads.”

Margarita Simonyan

23,936 views • 4 days ago

Theoretical physicist Michio Kaku on why aging is simply an accumulation of errors that AI can now identify: He opens with a bold claim about what artificial intelligence might finally make possible: "Artificial intelligence may give us something that the kings and queens of old could never conquer and that is the aging process." For Dr. Michio Kaku, the science behind why we age is no longer a mystery. It comes down to one thing: errors building up inside our cells over time. "We now know why things age. Things get old and die because they accumulate error. Error in the form of DNA mistakes. Accumulation of errors inside cells." The challenge, until now, has been the sheer scale of the problem. Identifying exactly where and how these errors occur across the human genome has been beyond what conventional research could handle. That's where artificial intelligence changes everything: "But now we have artificial intelligence. We can analyze millions of genomes of old people, compare them to millions of genomes of young people, and identify where aging takes place in the body."
0:42

Theoretical physicist Michio Kaku on why aging is simply an accumulation of errors that AI can now identify: He opens with a bold claim about what artificial intelligence might finally make possible: "Artificial intelligence may give us something that the kings and queens of old could never conquer and that is the aging process." For Dr. Michio Kaku, the science behind why we age is no longer a mystery. It comes down to one thing: errors building up inside our cells over time. "We now know why things age. Things get old and die because they accumulate error. Error in the form of DNA mistakes. Accumulation of errors inside cells." The challenge, until now, has been the sheer scale of the problem. Identifying exactly where and how these errors occur across the human genome has been beyond what conventional research could handle. That's where artificial intelligence changes everything: "But now we have artificial intelligence. We can analyze millions of genomes of old people, compare them to millions of genomes of young people, and identify where aging takes place in the body."

Big Brain AI

78,224 views • 1 month ago

So many people talking about AI evals. Hamel Husain and Shreya Shankar show me how to actually build one. Live. Hamel and Shreya teach the world’s most popular course on evals, and have long been at the forefront of this emerging and important new skill for AI product builders. I'm so excited to finally have them on the podcast. Learn: 🔸 WTF are evals 🔸 A step-by-step live walkthrough of how to create an eval 🔸 Why evals have become so important 🔸 The debate between “vibes” and systematic evals 🔸 Code-based evals vs. LLM-as-judge 🔸 Why you only need 4-7 evals (not hundreds) 🔸 Building an LLM judge in Google Sheets 🔸 Why you always want to start with error analysis 🔸 Much more Listen now 👇 • YouTube: • Spotify: • Apple: Thank you to our wonderful sponsors for supporting the podcast: 🏆 Fin — The #1 AI agent for customer service: 🏆 dscout — The UX platform to capture insights at every stage: from ideation to production: 🏆 Mercury — The art of simplified finances:
1:12

So many people talking about AI evals. Hamel Husain and Shreya Shankar show me how to actually build one. Live. Hamel and Shreya teach the world’s most popular course on evals, and have long been at the forefront of this emerging and important new skill for AI product builders. I'm so excited to finally have them on the podcast. Learn: 🔸 WTF are evals 🔸 A step-by-step live walkthrough of how to create an eval 🔸 Why evals have become so important 🔸 The debate between “vibes” and systematic evals 🔸 Code-based evals vs. LLM-as-judge 🔸 Why you only need 4-7 evals (not hundreds) 🔸 Building an LLM judge in Google Sheets 🔸 Why you always want to start with error analysis 🔸 Much more Listen now 👇 • YouTube: • Spotify: • Apple: Thank you to our wonderful sponsors for supporting the podcast: 🏆 Fin — The #1 AI agent for customer service: 🏆 dscout — The UX platform to capture insights at every stage: from ideation to production: 🏆 Mercury — The art of simplified finances:

Lenny Rachitsky

125,296 views • 9 months ago

As AI reshapes our world, we must ensure it benefits everyone – and that it is accessible and safe. The first global scientific body on artificial intelligence brings together independent experts to assess how AI is transforming our lives and how it can serve the common good.
0:52

As AI reshapes our world, we must ensure it benefits everyone – and that it is accessible and safe. The first global scientific body on artificial intelligence brings together independent experts to assess how AI is transforming our lives and how it can serve the common good.

United Nations

34,387 views • 9 days ago

"I think one of the truths is that the demand for AI intelligence and smarter AI intelligence is going to be insatiable. Some people disagree that once we reach certain thresholds then you don't need more intelligence. To that I ask people, 'Have you ever worked with or managed someone in a work environment and wouldn't press a button to make them smarter?'" George elaborates on his thesis that the demand for smarter AI intelligence will only increase over time even as models get more powerful. George Cameron
0:52

"I think one of the truths is that the demand for AI intelligence and smarter AI intelligence is going to be insatiable. Some people disagree that once we reach certain thresholds then you don't need more intelligence. To that I ask people, 'Have you ever worked with or managed someone in a work environment and wouldn't press a button to make them smarter?'" George elaborates on his thesis that the demand for smarter AI intelligence will only increase over time even as models get more powerful. George Cameron

Latent.Space

10,629 views • 5 months ago

How will artificial intelligence transform the way we live, work, and shape our lifestyles over the next decade? And why isn’t this revolution attracting the scrutiny it deserves? Join Rory Stewart and Matt Clifford for How AI Will Change The World, a brand-new TRIP+ exclusive mini-series exploring the opportunities, risks, and realities of the AI revolution. Episode one available now.
1:26

How will artificial intelligence transform the way we live, work, and shape our lifestyles over the next decade? And why isn’t this revolution attracting the scrutiny it deserves? Join Rory Stewart and Matt Clifford for How AI Will Change The World, a brand-new TRIP+ exclusive mini-series exploring the opportunities, risks, and realities of the AI revolution. Episode one available now.

The Rest Is Politics

22,485 views • 6 months ago

🚨NEW: We sat down with Anthony Noto, CEO of SoFi, to discuss how the company became the first nationally chartered bank in the U.S. to offer crypto trading and why it’s now launching a native stablecoin built on Ethereum. We cover SoFi’s long-term crypto strategy, efforts to lower borrowing costs, and how the bank is shaping the next phase of digital asset adoption through stablecoins and tokenized deposits. Anthony also shares insights on market structure in 2026, the intersection of AI and blockchain, and how SoFi is positioning itself for the future of crypto banking.
50:46

🚨NEW: We sat down with Anthony Noto, CEO of SoFi, to discuss how the company became the first nationally chartered bank in the U.S. to offer crypto trading and why it’s now launching a native stablecoin built on Ethereum. We cover SoFi’s long-term crypto strategy, efforts to lower borrowing costs, and how the bank is shaping the next phase of digital asset adoption through stablecoins and tokenized deposits. Anthony also shares insights on market structure in 2026, the intersection of AI and blockchain, and how SoFi is positioning itself for the future of crypto banking.

Crypto In America

149,535 views • 6 months ago

🤖The future is being built right now — and the UAE is leading the way by betting big on artificial intelligence. From public services to economic planning, AI is being embedded into national systems across the United Arab Emirates. The country is rapidly becoming an AI hub, through integrating AI into key sectors like healthcare, transport, finance and infrastructure. The leadership have launched the National AI Strategy 2031, aiming to become a global leader in artificial intelligence and have recognized one reality: AI will soon replace many jobs so, societies must adapt early.
0:38

🤖The future is being built right now — and the UAE is leading the way by betting big on artificial intelligence. From public services to economic planning, AI is being embedded into national systems across the United Arab Emirates. The country is rapidly becoming an AI hub, through integrating AI into key sectors like healthcare, transport, finance and infrastructure. The leadership have launched the National AI Strategy 2031, aiming to become a global leader in artificial intelligence and have recognized one reality: AI will soon replace many jobs so, societies must adapt early.

Oli London

22,807 views • 3 months ago

Artificial intelligence "is not imagining anything new, yet" says We ask the artist and founder of AI company, FYI whether he thinks AI is a threat to the music industry.
1:59

Artificial intelligence "is not imagining anything new, yet" says We ask the artist and founder of AI company, FYI whether he thinks AI is a threat to the music industry.

Channel 4 News

11,830 views • 1 year ago

Ready to shape the AI era? Join Huawei at MWC 2026 Barcelona for Advancing All Intelligence. Learn new AI trends from industry leaders and discover how IT infrastructure is evolving. It's time to explore the boundaries of intelligence and create a new architecture for the AI era.
0:39

Ready to shape the AI era? Join Huawei at MWC 2026 Barcelona for Advancing All Intelligence. Learn new AI trends from industry leaders and discover how IT infrastructure is evolving. It's time to explore the boundaries of intelligence and create a new architecture for the AI era.

Huawei

1,568,565 views • 3 months ago

“I care about one thing and one thing only: that the 18, 19, 20, year old kid—who had no choice in where he went, or what threat he was facing—I want him to win and come home. That’s why we do it. Palantir is very helpful in delivering this.” Cameron Stanley, Chief Digital and Artificial Intelligence Officer of the Department of War, shares how the DoW is driving enterprise-wide adoption of data, analytics, and AI to generate decision advantage—and what it takes to move cutting-edge technology from the lab to the warfighter at speed.
1:04

“I care about one thing and one thing only: that the 18, 19, 20, year old kid—who had no choice in where he went, or what threat he was facing—I want him to win and come home. That’s why we do it. Palantir is very helpful in delivering this.” Cameron Stanley, Chief Digital and Artificial Intelligence Officer of the Department of War, shares how the DoW is driving enterprise-wide adoption of data, analytics, and AI to generate decision advantage—and what it takes to move cutting-edge technology from the lab to the warfighter at speed.

Palantir

443,119 views • 3 months ago

Ontologies, Evals & Deployed Intelligence: Building Real AI Agents - Mitchell Troyanovsky, co-founder of Basis 0:04: Intro /Meet Mitch Troyanovsky 2:20: What is an AI Agent, actually? 4:07: The Gap Between LLMs and Embodied Intelligence 5:25: Why Ontology is the Missing Piece of Agent Design 8:10: Evals, Evals, Evals 9:57: Building a Deployed Intelligence Team 11:35: The Path to Being a Good AI Agent Builder From our most recent #DataDrivenNYC
13:01

Ontologies, Evals & Deployed Intelligence: Building Real AI Agents - Mitchell Troyanovsky, co-founder of Basis 0:04: Intro /Meet Mitch Troyanovsky 2:20: What is an AI Agent, actually? 4:07: The Gap Between LLMs and Embodied Intelligence 5:25: Why Ontology is the Missing Piece of Agent Design 8:10: Evals, Evals, Evals 9:57: Building a Deployed Intelligence Team 11:35: The Path to Being a Good AI Agent Builder From our most recent #DataDrivenNYC

Matt Turck

10,969 views • 9 months ago

François Chollet (François Chollet) has spent years asking a different question than most of the AI world. Instead of scaling what already works, he’s trying to understand what intelligence actually is and how to build it from first principles. In this episode of the Lightcone Podcast, he traces that path from his early work on deep learning to the creation of the ARC Prize, and the launch of ARC V3, a new benchmark designed to measure something deeper than performance: the ability to learn, adapt, and reason efficiently in entirely new environments. He explains why today’s systems may be hitting limits, what recent breakthroughs really mean, and why reaching true general intelligence may require a fundamentally different approach. 00:00 - AGI by 2030? 00:31 - Introducing Ndea: A New Path Beyond Deep Learning 01:08 - A New ML Paradigm 01:30 - Replacing neural nets with compact symbolic programs 03:04 - Why Ndea Isn’t Competing With Coding Agents 05:20 - Why Everyone Might Be Wrong About Scaling LLMs 07:22 - Why Coding Agents Suddenly Work So Well 08:50 - The Limits of LLMs in Non-Verifiable Domains 10:48 - What AGI Actually Means (And Why Most Definitions Are Wrong) 13:30 - Why Deep Learning Hits a Wall 14:00 - ARC’s Origin Story 18:20 - ARC Benchmarks Explained: From V1 to V3 22:49 - The RL Loop Powering Coding Agents Today 27:03 - ARC-AGI V3: Measuring “Agentic Intelligence” 31:14 - Inside the ARC Game Studio 35:31 - Could AGI Fit in 10,000 Lines of Code? 44:01 - Building Ndea: From Idea to Compounding Research Stack 46:46 - The Future of ARC: Benchmarks That Evolve With AI 47:21 - Why There’s Still Huge Opportunity for New AI Paradigms 53:37 - How to Build a Breakout Open Source Project - Lessons From Keras 56:39 - Advice For How To Think About AI
57:23

François Chollet (François Chollet) has spent years asking a different question than most of the AI world. Instead of scaling what already works, he’s trying to understand what intelligence actually is and how to build it from first principles. In this episode of the Lightcone Podcast, he traces that path from his early work on deep learning to the creation of the ARC Prize, and the launch of ARC V3, a new benchmark designed to measure something deeper than performance: the ability to learn, adapt, and reason efficiently in entirely new environments. He explains why today’s systems may be hitting limits, what recent breakthroughs really mean, and why reaching true general intelligence may require a fundamentally different approach. 00:00 - AGI by 2030? 00:31 - Introducing Ndea: A New Path Beyond Deep Learning 01:08 - A New ML Paradigm 01:30 - Replacing neural nets with compact symbolic programs 03:04 - Why Ndea Isn’t Competing With Coding Agents 05:20 - Why Everyone Might Be Wrong About Scaling LLMs 07:22 - Why Coding Agents Suddenly Work So Well 08:50 - The Limits of LLMs in Non-Verifiable Domains 10:48 - What AGI Actually Means (And Why Most Definitions Are Wrong) 13:30 - Why Deep Learning Hits a Wall 14:00 - ARC’s Origin Story 18:20 - ARC Benchmarks Explained: From V1 to V3 22:49 - The RL Loop Powering Coding Agents Today 27:03 - ARC-AGI V3: Measuring “Agentic Intelligence” 31:14 - Inside the ARC Game Studio 35:31 - Could AGI Fit in 10,000 Lines of Code? 44:01 - Building Ndea: From Idea to Compounding Research Stack 46:46 - The Future of ARC: Benchmarks That Evolve With AI 47:21 - Why There’s Still Huge Opportunity for New AI Paradigms 53:37 - How to Build a Breakout Open Source Project - Lessons From Keras 56:39 - Advice For How To Think About AI

Y Combinator

151,120 views • 2 months ago

We built Lit Protocol 🔑 to usher in a new paradigm of digital identity that empowers individuals to maintain sovereignty in the AI era. This infrastructure is crucial as we navigate the complex intersection of human agency and artificial intelligence.
2:05

We built Lit Protocol 🔑 to usher in a new paradigm of digital identity that empowers individuals to maintain sovereignty in the AI era. This infrastructure is crucial as we navigate the complex intersection of human agency and artificial intelligence.

David Sneider 🎆

19,640 views • 1 year ago

ARC-AGI is redefining how to measure progress on the path to AGI - focusing on reasoning, generalization, and adaptability instead of memorization or scale. At NeurIPS 2025, YC's Diana sat down with ARC Prize President Greg Kamradt to find out why most AI benchmarks fail, how ARC-AGI reveals the limits of today’s models, and why measuring intelligence may be harder than building it. 00:11 — What ARC Prize is and why it exists 00:38 — François Chollet’s definition of AGI 01:48 — What ARC-AGI Actually Tests 02:25 — When LLMs Failed the ARC Benchmark 03:38 — ARC-AGI Becomes the Standard 04:49 — False Positives in AI Progress 06:06 — The Evolution of ARC-AGI 08:55 — Measuring Intelligence beyond just accuracy 10:25 — What happens if a model solves ARC-AGI?
11:59

ARC-AGI is redefining how to measure progress on the path to AGI - focusing on reasoning, generalization, and adaptability instead of memorization or scale. At NeurIPS 2025, YC's Diana sat down with ARC Prize President Greg Kamradt to find out why most AI benchmarks fail, how ARC-AGI reveals the limits of today’s models, and why measuring intelligence may be harder than building it. 00:11 — What ARC Prize is and why it exists 00:38 — François Chollet’s definition of AGI 01:48 — What ARC-AGI Actually Tests 02:25 — When LLMs Failed the ARC Benchmark 03:38 — ARC-AGI Becomes the Standard 04:49 — False Positives in AI Progress 06:06 — The Evolution of ARC-AGI 08:55 — Measuring Intelligence beyond just accuracy 10:25 — What happens if a model solves ARC-AGI?

Y Combinator

98,369 views • 6 months ago

On a new edition of The Four Horsemen, MonteCristo and Thorin lay out the historical context of co-streaming, trace the full arc of how esports co-streaming evolved, and share why Riot Games is to blame for many of the current problems. Streaming everywhere now, links below!
2:08:20

On a new edition of The Four Horsemen, MonteCristo and Thorin lay out the historical context of co-streaming, trace the full arc of how esports co-streaming evolved, and share why Riot Games is to blame for many of the current problems. Streaming everywhere now, links below!

Last Free Nation

14,072 views • 1 month ago

A fascinating conversation with the former CIA Director for Russia Analysis, George Beebe. How will the U.S. adjust to a new world order, and why is Europe failing so spectacularly?
32:59

A fascinating conversation with the former CIA Director for Russia Analysis, George Beebe. How will the U.S. adjust to a new world order, and why is Europe failing so spectacularly?

Glenn Diesen

25,236 views • 5 months ago

The rise of AI factories marks a new industrial era that is built on efficiency, intelligence, and sustainability. How we build them will shape economies, industries, and the future of the planet.
0:27

The rise of AI factories marks a new industrial era that is built on efficiency, intelligence, and sustainability. How we build them will shape economies, industries, and the future of the planet.

NVIDIA Newsroom

56,120 views • 1 year ago