Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

ARC-AGI-3 Preview Event Recap Greg Kamradt steps through our Interactive Reasoning Benchmark thesis * Why static benchmarks fall short measuring agentic capabilities * The ARC Prize approach to creating interactive benchmarks

arcprize's profile picture

ARC Prize

28,225 subscribers

22,248 Aufrufe • vor 11 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Interactive Reasoning Benchmarks are the next step in frontier evaluations Hear Greg Kamradt share why measuring human-like intelligence requires multi-turn environments Including a sneak peak of ARC-AGI-3 Want to help us build interactive evaluations? We're hiring
18:14

Interactive Reasoning Benchmarks are the next step in frontier evaluations Hear Greg Kamradt share why measuring human-like intelligence requires multi-turn environments Including a sneak peak of ARC-AGI-3 Want to help us build interactive evaluations? We're hiring

ARC Prize

26,218 Aufrufe • vor 1 Jahr

ARC-AGI is redefining how to measure progress on the path to AGI - focusing on reasoning, generalization, and adaptability instead of memorization or scale. At NeurIPS 2025, YC's Diana sat down with ARC Prize President Greg Kamradt to find out why most AI benchmarks fail, how ARC-AGI reveals the limits of today’s models, and why measuring intelligence may be harder than building it. 00:11 — What ARC Prize is and why it exists 00:38 — François Chollet’s definition of AGI 01:48 — What ARC-AGI Actually Tests 02:25 — When LLMs Failed the ARC Benchmark 03:38 — ARC-AGI Becomes the Standard 04:49 — False Positives in AI Progress 06:06 — The Evolution of ARC-AGI 08:55 — Measuring Intelligence beyond just accuracy 10:25 — What happens if a model solves ARC-AGI?
11:59

ARC-AGI is redefining how to measure progress on the path to AGI - focusing on reasoning, generalization, and adaptability instead of memorization or scale. At NeurIPS 2025, YC's Diana sat down with ARC Prize President Greg Kamradt to find out why most AI benchmarks fail, how ARC-AGI reveals the limits of today’s models, and why measuring intelligence may be harder than building it. 00:11 — What ARC Prize is and why it exists 00:38 — François Chollet’s definition of AGI 01:48 — What ARC-AGI Actually Tests 02:25 — When LLMs Failed the ARC Benchmark 03:38 — ARC-AGI Becomes the Standard 04:49 — False Positives in AI Progress 06:06 — The Evolution of ARC-AGI 08:55 — Measuring Intelligence beyond just accuracy 10:25 — What happens if a model solves ARC-AGI?

Y Combinator

98,369 Aufrufe • vor 6 Monaten

ARC-AGI-3 Launch Presentation Greg Kamradt launches ARC-AGI-3 on March 25, 2026 in San Francisco Including: - ARC-AGI as a signal for external progress - ARC-AGI-3 Game design process - Announcement of ARC Prize 2026 Competition
14:52

ARC-AGI-3 Launch Presentation Greg Kamradt launches ARC-AGI-3 on March 25, 2026 in San Francisco Including: - ARC-AGI as a signal for external progress - ARC-AGI-3 Game design process - Announcement of ARC Prize 2026 Competition

ARC Prize

20,097 Aufrufe • vor 3 Monaten

ARC Prize President Greg Kamradt says OpenAI's o3 model has surpassed the human threshold on the ARC-AGI benchmark - scoring 87.5 where the human threshold is 85 in a "major milestone" that is "new territory" in the ARC-AGI world
1:03

ARC Prize President Greg Kamradt says OpenAI's o3 model has surpassed the human threshold on the ARC-AGI benchmark - scoring 87.5 where the human threshold is 85 in a "major milestone" that is "new territory" in the ARC-AGI world

Tsarathustra

71,466 Aufrufe • vor 1 Jahr

François Chollet (François Chollet) on the ARC Prize and how we get to AGI. At AI Startup School in San Francisco. 00:00 - The Falling Cost of Compute 00:57 - Deep-Learning’s Scaling Era & Benchmarks 01:59 - The ARC Benchmark 03:02 - The 2024 Shift to Test-Time Adaptation 05:01 - What Is Intelligence? 07:12 - Why Benchmarks Matter (and Mislead) 08:57 - ARC 1 Exposes Scaling Limits 10:58 - ARC 2: Compositional Reasoning Arrives 12:55 - Humans vs. Models on ARC2 14:58 - Previewing ARC3 & Interactive Agency 17:00 - Kaleidoscopic Hypothesis and Abstractions 22:00 - Type 1 vs. Type 2 Abstractions 26:00 - Discrete Program Search & Inventive AI 29:00 - Fusing Intuition with Symbolic Reasoning 32:00 - Building AGI Through Meta-Learning Systems
34:47

François Chollet (François Chollet) on the ARC Prize and how we get to AGI. At AI Startup School in San Francisco. 00:00 - The Falling Cost of Compute 00:57 - Deep-Learning’s Scaling Era & Benchmarks 01:59 - The ARC Benchmark 03:02 - The 2024 Shift to Test-Time Adaptation 05:01 - What Is Intelligence? 07:12 - Why Benchmarks Matter (and Mislead) 08:57 - ARC 1 Exposes Scaling Limits 10:58 - ARC 2: Compositional Reasoning Arrives 12:55 - Humans vs. Models on ARC2 14:58 - Previewing ARC3 & Interactive Agency 17:00 - Kaleidoscopic Hypothesis and Abstractions 22:00 - Type 1 vs. Type 2 Abstractions 26:00 - Discrete Program Search & Inventive AI 29:00 - Fusing Intuition with Symbolic Reasoning 32:00 - Building AGI Through Meta-Learning Systems

Y Combinator

231,725 Aufrufe • vor 1 Jahr

CEO Dr. Ben Goertzel discusses AGI benchmarks and explains why a system passing François Chollet's Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) wouldn’t necessarily qualify as human-level AGI.
2:19

CEO Dr. Ben Goertzel discusses AGI benchmarks and explains why a system passing François Chollet's Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) wouldn’t necessarily qualify as human-level AGI.

SingularityNET

26,943 Aufrufe • vor 1 Jahr

.ARC Prize launched ARC AGI 3. AI scores less than 1%, humans get 100%. Is this the ultimate benchmark?
9:37

.ARC Prize launched ARC AGI 3. AI scores less than 1%, humans get 100%. Is this the ultimate benchmark?

Matthew Berman

26,969 Aufrufe • vor 3 Monaten

OpenAI's new o3 reasoning model has achieved a verified state-of-the-art score on the ARC-AGI benchmark
1:06

OpenAI's new o3 reasoning model has achieved a verified state-of-the-art score on the ARC-AGI benchmark

Tsarathustra

27,655 Aufrufe • vor 1 Jahr

Are AI benchmarks doomed? Greg Burnham and Tom Adamczewski join Anson Ho to push back on benchmark pessimism and dig into what the next generation of AI benchmarks could look like. (0:00:00) - Preview (0:00:36) - Intro: Are AI benchmarks doomed? (0:03:13) - The costs and benefits of benchmark development (0:11:48) - MirrorCode and scalable benchmarks (0:20:57) - AI speed-up in benchmark development (0:23:28) - The benchmark-reality gap (0:38:26) - Can an AGI benchmark exist? (0:43:18) - Beyond automated scoring (1:00:45) - How AI changes benchmark building in practice
1:05:24

Are AI benchmarks doomed? Greg Burnham and Tom Adamczewski join Anson Ho to push back on benchmark pessimism and dig into what the next generation of AI benchmarks could look like. (0:00:00) - Preview (0:00:36) - Intro: Are AI benchmarks doomed? (0:03:13) - The costs and benefits of benchmark development (0:11:48) - MirrorCode and scalable benchmarks (0:20:57) - AI speed-up in benchmark development (0:23:28) - The benchmark-reality gap (0:38:26) - Can an AGI benchmark exist? (0:43:18) - Beyond automated scoring (1:00:45) - How AI changes benchmark building in practice

Epoch AI

22,037 Aufrufe • vor 2 Monaten

I'm partnering with Mike Knoop to launch ARC Prize: a $1,000,000 competition to create an AI that can adapt to novelty and solve simple reasoning problems. Let's get back on track towards AGI. Website: ARC Prize on Kaggle:
0:35

I'm partnering with Mike Knoop to launch ARC Prize: a $1,000,000 competition to create an AI that can adapt to novelty and solve simple reasoning problems. Let's get back on track towards AGI. Website: ARC Prize on Kaggle:

François Chollet

602,375 Aufrufe • vor 2 Jahren

François Chollet (François Chollet) has spent years asking a different question than most of the AI world. Instead of scaling what already works, he’s trying to understand what intelligence actually is and how to build it from first principles. In this episode of the Lightcone Podcast, he traces that path from his early work on deep learning to the creation of the ARC Prize, and the launch of ARC V3, a new benchmark designed to measure something deeper than performance: the ability to learn, adapt, and reason efficiently in entirely new environments. He explains why today’s systems may be hitting limits, what recent breakthroughs really mean, and why reaching true general intelligence may require a fundamentally different approach. 00:00 - AGI by 2030? 00:31 - Introducing Ndea: A New Path Beyond Deep Learning 01:08 - A New ML Paradigm 01:30 - Replacing neural nets with compact symbolic programs 03:04 - Why Ndea Isn’t Competing With Coding Agents 05:20 - Why Everyone Might Be Wrong About Scaling LLMs 07:22 - Why Coding Agents Suddenly Work So Well 08:50 - The Limits of LLMs in Non-Verifiable Domains 10:48 - What AGI Actually Means (And Why Most Definitions Are Wrong) 13:30 - Why Deep Learning Hits a Wall 14:00 - ARC’s Origin Story 18:20 - ARC Benchmarks Explained: From V1 to V3 22:49 - The RL Loop Powering Coding Agents Today 27:03 - ARC-AGI V3: Measuring “Agentic Intelligence” 31:14 - Inside the ARC Game Studio 35:31 - Could AGI Fit in 10,000 Lines of Code? 44:01 - Building Ndea: From Idea to Compounding Research Stack 46:46 - The Future of ARC: Benchmarks That Evolve With AI 47:21 - Why There’s Still Huge Opportunity for New AI Paradigms 53:37 - How to Build a Breakout Open Source Project - Lessons From Keras 56:39 - Advice For How To Think About AI
57:23

François Chollet (François Chollet) has spent years asking a different question than most of the AI world. Instead of scaling what already works, he’s trying to understand what intelligence actually is and how to build it from first principles. In this episode of the Lightcone Podcast, he traces that path from his early work on deep learning to the creation of the ARC Prize, and the launch of ARC V3, a new benchmark designed to measure something deeper than performance: the ability to learn, adapt, and reason efficiently in entirely new environments. He explains why today’s systems may be hitting limits, what recent breakthroughs really mean, and why reaching true general intelligence may require a fundamentally different approach. 00:00 - AGI by 2030? 00:31 - Introducing Ndea: A New Path Beyond Deep Learning 01:08 - A New ML Paradigm 01:30 - Replacing neural nets with compact symbolic programs 03:04 - Why Ndea Isn’t Competing With Coding Agents 05:20 - Why Everyone Might Be Wrong About Scaling LLMs 07:22 - Why Coding Agents Suddenly Work So Well 08:50 - The Limits of LLMs in Non-Verifiable Domains 10:48 - What AGI Actually Means (And Why Most Definitions Are Wrong) 13:30 - Why Deep Learning Hits a Wall 14:00 - ARC’s Origin Story 18:20 - ARC Benchmarks Explained: From V1 to V3 22:49 - The RL Loop Powering Coding Agents Today 27:03 - ARC-AGI V3: Measuring “Agentic Intelligence” 31:14 - Inside the ARC Game Studio 35:31 - Could AGI Fit in 10,000 Lines of Code? 44:01 - Building Ndea: From Idea to Compounding Research Stack 46:46 - The Future of ARC: Benchmarks That Evolve With AI 47:21 - Why There’s Still Huge Opportunity for New AI Paradigms 53:37 - How to Build a Breakout Open Source Project - Lessons From Keras 56:39 - Advice For How To Think About AI

Y Combinator

151,141 Aufrufe • vor 3 Monaten

ARC Prize 2025 Winners Interviews Paper Award 3rd Place Isaac Liao shares the story behind CompressARC - an MDL-based, single puzzle-trained neural code golf system that achieves ~20–34% on ARC-AGI-1 and ~4% on ARC-AGI-2 without any pretraining or external data.
37:50

ARC Prize 2025 Winners Interviews Paper Award 3rd Place Isaac Liao shares the story behind CompressARC - an MDL-based, single puzzle-trained neural code golf system that achieves ~20–34% on ARC-AGI-1 and ~4% on ARC-AGI-2 without any pretraining or external data.

ARC Prize

55,608 Aufrufe • vor 7 Monaten

Gemini 3 Flash can analyze high-fidelity images and use multimodal reasoning to determine next steps. See how the model understands complex visuals and generates layers of interactive elements.
0:28

Gemini 3 Flash can analyze high-fidelity images and use multimodal reasoning to determine next steps. See how the model understands complex visuals and generates layers of interactive elements.

Google AI Developers

47,280 Aufrufe • vor 6 Monaten

"I've updated my AGI timeline." One year later, Dwarkesh Patel and François Chollet meet on camera again. Both of them have shifted their AGI timelines. They dive into AGI macroeconomics, the singularity, and ARC-AGI-3 preview.
23:46

"I've updated my AGI timeline." One year later, Dwarkesh Patel and François Chollet meet on camera again. Both of them have shifted their AGI timelines. They dive into AGI macroeconomics, the singularity, and ARC-AGI-3 preview.

ARC Prize

47,455 Aufrufe • vor 11 Monaten

We interviewed the official winners of the ARC Prize - the ARChitects. Their approach was extremely creative! On MLST now.
0:40

We interviewed the official winners of the ARC Prize - the ARChitects. Their approach was extremely creative! On MLST now.

Machine Learning Street Talk

17,593 Aufrufe • vor 1 Jahr

Introducing Gemini 3.1 Pro 🚀 3.1 Pro represents a major step forward in core reasoning. It scored 77.1% (more than doubling 3 Pro’s score) on ARC-AGI-2, the benchmark that evaluates a model's ability to solve new logic patterns and work through challenges it hasn’t encountered before. This demo illustrates how the model can go beyond the prompt. Instead of rendering a video or static graphic, 3.1 Pro codes a full environment, integrating generative audio and providing UI controls.
0:47

Introducing Gemini 3.1 Pro 🚀 3.1 Pro represents a major step forward in core reasoning. It scored 77.1% (more than doubling 3 Pro’s score) on ARC-AGI-2, the benchmark that evaluates a model's ability to solve new logic patterns and work through challenges it hasn’t encountered before. This demo illustrates how the model can go beyond the prompt. Instead of rendering a video or static graphic, 3.1 Pro codes a full environment, integrating generative audio and providing UI controls.

Google AI

3,067,002 Aufrufe • vor 4 Monaten

Francois Chollet + Sam Altman Fireside François Chollet and Sam Altman fireside during ARC-AGI-3 Launch Party moderated by Deedy They discuss: - Social contracts evolving - AGI views as a parent - When will labs score >85% on ARC-AGI-3?
29:02

Francois Chollet + Sam Altman Fireside François Chollet and Sam Altman fireside during ARC-AGI-3 Launch Party moderated by Deedy They discuss: - Social contracts evolving - AGI views as a parent - When will labs score >85% on ARC-AGI-3?

ARC Prize

33,317 Aufrufe • vor 3 Monaten

Just tried ChatGPT agent on ARC Prize ARC-AGI-3 > Told it to play a game > Couldn't figure out what to do > Agent did a web search, "how to beat X arc-agi-3 game" > It didn't find answers > I told it to try clicking red/blue blocks > It clicked them, noticed something happened, kept clicking > Nudged more > Couldn't figure it out > Searched again Then I cut it off btw agent is a very cool tool
0:56

Just tried ChatGPT agent on ARC Prize ARC-AGI-3 > Told it to play a game > Couldn't figure out what to do > Agent did a web search, "how to beat X arc-agi-3 game" > It didn't find answers > I told it to try clicking red/blue blocks > It clicked them, noticed something happened, kept clicking > Nudged more > Couldn't figure it out > Searched again Then I cut it off btw agent is a very cool tool

Greg Kamradt

15,386 Aufrufe • vor 11 Monaten