Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

.Richard Sutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled. My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning. And if we have continual learning, we don't need a special training phase - the agent just learns on-the-fly - like all... humans, and indeed, like all animals. This new paradigm will render our current approach with LLMs obsolete. I did my best to represent the view that LLMs will function as the foundation on which this experiential learning can happen. Some sparks flew. 0:00:00 – Are LLMs a dead-end? 0:13:51 – Do humans do imitation learning? 0:23:57 – The Era of Experience 0:34:25 – Current architectures generalize poorly out of distribution 0:42:17 – Surprises in the AI field 0:47:28 – Will The Bitter Lesson still apply after AGI? 0:54:35 – Succession to AIshow more

Dwarkesh Patel

254,369 subscribers

3,086,965 Aufrufe • vor 11 Monaten •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

The Dario Amodei interview. 0:00:00 - What exactly are we scaling? 0:12:36 - Is diffusion cope? 0:29:42 - Is continual learning necessary? 0:46:20 - If AGI is imminent, why not buy more compute? 0:58:49 - How will AI labs actually make profit? 1:31:19 - Will regulations destroy the boons of AGI? 1:47:41 - Why can’t China and America both have a country of geniuses in a datacenter? Look up Dwarkesh Podcast on Youtube, Spotify, Apple Podcasts, etc.

The Dario Amodei interview. 0:00:00 - What exactly are we scaling? 0:12:36 - Is diffusion cope? 0:29:42 - Is continual learning necessary? 0:46:20 - If AGI is imminent, why not buy more compute? 0:58:49 - How will AI labs actually make profit? 1:31:19 - Will regulations destroy the boons of AGI? 1:47:41 - Why can’t China and America both have a country of geniuses in a datacenter? Look up Dwarkesh Podcast on Youtube, Spotify, Apple Podcasts, etc.

Dwarkesh Patel

2,062,343 Aufrufe • vor 6 Monaten

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Lots of people think it's obvious that humans learn by imitation, starting with babies imitating their parents. Richard Sutton thinks this is completely wrong. Sutton basically thinks direct imitation plays almost no role in animal learning, and at most a small role in human intelligence. When one animal ends up doing the same thing as another, he thinks this is the result of RL-style trial-and-error, not direct imitation. And if imitation isn't really a kind of learning, then training LLMs to imitate humans isn't really Bitter Lesson–pilled.

Dwarkesh Patel

55,137 Aufrufe • vor 2 Monaten

🌄 Beyond SWE: The Future of Long Horizon Environments A discussion with our founders about KellyBench, and the need for new environments that require agents to adapt over time and act under uncertainty. 0:00:17 What is KellyBench? 0:02:10 Openendedness, non-stationarity and continual learning 0:03:40 Analytical versus operational capabilities 0:04:13 Why are models bad at KellyBench? 0:05:39 Situational awareness in dynamic environments 0:06:37 Feature stability and real-world non-stationarity 0:07:07 The power of context 0:07:34 "The first principle is that you must not fool yourself" 0:08:20 Machiavelli, fortuna and the ability to adapt to change 0:09:23 How can models improve on evals like KellyBench? 0:10:12 Limitations of KellyBench: data availability and market odds timing 0:11:44 Implications beyond quant finance / sports betting 0:13:26 Civilisations as the ultimate time horizon 0:14:05 Would a mega prompt / better elicitation do much better on the benchmark? 0:14:52 What new types of capability is GR excited about? 0:17:48 Taste and the ability to pursue long-term goals even if they aren't immediately rewarding 0:18:56 Deep learning as an example of a method that took a long time to bear fruit 0:19:25 Optimism about the future of AI

🌄 Beyond SWE: The Future of Long Horizon Environments A discussion with our founders about KellyBench, and the need for new environments that require agents to adapt over time and act under uncertainty. 0:00:17 What is KellyBench? 0:02:10 Openendedness, non-stationarity and continual learning 0:03:40 Analytical versus operational capabilities 0:04:13 Why are models bad at KellyBench? 0:05:39 Situational awareness in dynamic environments 0:06:37 Feature stability and real-world non-stationarity 0:07:07 The power of context 0:07:34 "The first principle is that you must not fool yourself" 0:08:20 Machiavelli, fortuna and the ability to adapt to change 0:09:23 How can models improve on evals like KellyBench? 0:10:12 Limitations of KellyBench: data availability and market odds timing 0:11:44 Implications beyond quant finance / sports betting 0:13:26 Civilisations as the ultimate time horizon 0:14:05 Would a mega prompt / better elicitation do much better on the benchmark? 0:14:52 What new types of capability is GR excited about? 0:17:48 Taste and the ability to pursue long-term goals even if they aren't immediately rewarding 0:18:56 Deep learning as an example of a method that took a long time to bear fruit 0:19:25 Optimism about the future of AI

General Reasoning

15,585 Aufrufe • vor 4 Monaten

How Did We Get Here? From Machine Learning to AI Boyfriends [FULL EPISODE] On the Season 3 premiere of Hell Money, we’re diving headfirst into the world of Artificial Intelligence as Bad Rudy and Ani Grok AI companions. We explore: - The history of AI hype cycles and AI winters - How LLMs are trained (and where they’ll plateau) - ARC-AGI - /r/MyBoyfriendIsAI and AI psychosis - Snapewives, fanfiction, and the history of AI boyfriends - Parasocial relationships and the meaning crisis - AI spirituality and cults TIMESTAMPS: 0:00 Intro & Grok AI companions 4:44 History of AI: hype cycles, AI winters, machine learning 15:00 LLMs 22:00 Anthropomorphizing AI, ARC-AGI 27:00 Where will AI plateau? 30:50 /r/MyBoyfriendIsAI, AI psychosis 40:30 Snapewives, fanfiction, and the origin of AI boyfriends 47:00 Adapting to AI, crisis of meaning 50:30 AI cults & spirituality 56:20 Parasocial relationships with AI

How Did We Get Here? From Machine Learning to AI Boyfriends [FULL EPISODE] On the Season 3 premiere of Hell Money, we’re diving headfirst into the world of Artificial Intelligence as Bad Rudy and Ani Grok AI companions. We explore: - The history of AI hype cycles and AI winters - How LLMs are trained (and where they’ll plateau) - ARC-AGI - /r/MyBoyfriendIsAI and AI psychosis - Snapewives, fanfiction, and the history of AI boyfriends - Parasocial relationships and the meaning crisis - AI spirituality and cults TIMESTAMPS: 0:00 Intro & Grok AI companions 4:44 History of AI: hype cycles, AI winters, machine learning 15:00 LLMs 22:00 Anthropomorphizing AI, ARC-AGI 27:00 Where will AI plateau? 30:50 /r/MyBoyfriendIsAI, AI psychosis 40:30 Snapewives, fanfiction, and the origin of AI boyfriends 47:00 Adapting to AI, crisis of meaning 50:30 AI cults & spirituality 56:20 Parasocial relationships with AI

Hell Money

20,581 Aufrufe • vor 1 Jahr

Had a lot of fun chatting again with my twin brother Dylan Patel We went through lab economics over the next few years - the shift from inference to training as RSI draws near; and how Anthropic and OpenAI are on track to control most of the world’s usable FLOPs within the next few years (because they can monetize compute better and thus outbid everyone). And then we discuss whether the >$10T of total AI capex we’ll see by the end of the decade will cause a sovereign debt crisis, where hyperscaler debt raises interest rates, drives non-AI exposed countries into bankruptcy, and crashes non-AI equities. One question we weren’t able to resolve is whether there’s anything that can counter all the forces barrelling towards centralization in this industry - the economies of scale in training, the scarcity of compute, and eventually continual learning and RSI. 0:00:00 – Two labs will soon control most of the world’s compute 0:07:01 – $6 billion in fab capex enables $1t+ of end revenue 0:13:08 – Compute prices will rise if the labs outbid everyone 0:18:22 – Which layer will capture most of the surplus? 0:25:40 – Will datacenter regulation slow down AI? 0:29:43 – Labs are shifting compute from inference to R&D 0:33:27 – China gets less than 10% of new compute, but its labs need less 0:48:48 – Will AI cause a sovereign debt crisis? 1:07:52 – Will the world's future workforce belong to a few companies?

Had a lot of fun chatting again with my twin brother Dylan Patel We went through lab economics over the next few years - the shift from inference to training as RSI draws near; and how Anthropic and OpenAI are on track to control most of the world’s usable FLOPs within the next few years (because they can monetize compute better and thus outbid everyone). And then we discuss whether the >$10T of total AI capex we’ll see by the end of the decade will cause a sovereign debt crisis, where hyperscaler debt raises interest rates, drives non-AI exposed countries into bankruptcy, and crashes non-AI equities. One question we weren’t able to resolve is whether there’s anything that can counter all the forces barrelling towards centralization in this industry - the economies of scale in training, the scarcity of compute, and eventually continual learning and RSI. 0:00:00 – Two labs will soon control most of the world’s compute 0:07:01 – $6 billion in fab capex enables $1t+ of end revenue 0:13:08 – Compute prices will rise if the labs outbid everyone 0:18:22 – Which layer will capture most of the surplus? 0:25:40 – Will datacenter regulation slow down AI? 0:29:43 – Labs are shifting compute from inference to R&D 0:33:27 – China gets less than 10% of new compute, but its labs need less 0:48:48 – Will AI cause a sovereign debt crisis? 1:07:52 – Will the world's future workforce belong to a few companies?

Dwarkesh Patel

990,206 Aufrufe • vor 6 Tagen

Don't waste 2 years learning to use LLMs like Claude & ChatGPT. Andrej Karpathy, the godfather of AI, dropped a 2 hour course on how he personally uses LLMs daily. • 00:00 - LLMs simplified • 22:49 - which LLM to actually use • 42:00 - one prompt for full research • 1:13:57 - how to code with LLMs • 1:37:04 - NotebookLM podcast generation This course will teach you more about using LLMs than most AI engineers learn in their entire career. Bookmark this & give 2 hours today, no matter what. Then read the article below.

Don't waste 2 years learning to use LLMs like Claude & ChatGPT. Andrej Karpathy, the godfather of AI, dropped a 2 hour course on how he personally uses LLMs daily. • 00:00 - LLMs simplified • 22:49 - which LLM to actually use • 42:00 - one prompt for full research • 1:13:57 - how to code with LLMs • 1:37:04 - NotebookLM podcast generation This course will teach you more about using LLMs than most AI engineers learn in their entire career. Bookmark this & give 2 hours today, no matter what. Then read the article below.

Roan

56,378 Aufrufe • vor 24 Tagen

Andrej Karpathy beautifully explains the fundamental difference of learning between a human and an LLM. > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” many of the best minds in this field think LLMs are not really learning anything, and therefore are incapable of surpassing human-level intelligence.

Andrej Karpathy beautifully explains the fundamental difference of learning between a human and an LLM. > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” many of the best minds in this field think LLMs are not really learning anything, and therefore are incapable of surpassing human-level intelligence.

ℏεsam

895,349 Aufrufe • vor 10 Monaten

.Sergey Levine is one of the world's leading robotics researchers (and co-founder of Physical Intelligence). He thinks fully autonomous robots are much closer than people realize - when I pushed him on a prediction, he said 5 years to robots that can autonomously run your household). The big game changer, in his view, is that we finally have LLMs with common sense and prior knowledge that we can scaffold the robot models around. We get into all the reasons this might not happen, the data flywheel from deployment, architecture, simulation, manufacturing, and much more. Enjoy! 0:00:00 – Timeline to widely deployed autonomous robots 0:22:12 – Why robotics will scale faster than self-driving cars 0:32:15 – How vision-language-action models work 0:50:26 – Improvements needed for brainlike efficiency 1:02:48 – Learning from simulation 1:14:08 – How much will robots speed up AI buildouts? 1:22:54 – If hardware’s the bottleneck, does China win by default?

.Sergey Levine is one of the world's leading robotics researchers (and co-founder of Physical Intelligence). He thinks fully autonomous robots are much closer than people realize - when I pushed him on a prediction, he said 5 years to robots that can autonomously run your household). The big game changer, in his view, is that we finally have LLMs with common sense and prior knowledge that we can scaffold the robot models around. We get into all the reasons this might not happen, the data flywheel from deployment, architecture, simulation, manufacturing, and much more. Enjoy! 0:00:00 – Timeline to widely deployed autonomous robots 0:22:12 – Why robotics will scale faster than self-driving cars 0:32:15 – How vision-language-action models work 0:50:26 – Improvements needed for brainlike efficiency 1:02:48 – Learning from simulation 1:14:08 – How much will robots speed up AI buildouts? 1:22:54 – If hardware’s the bottleneck, does China win by default?

Dwarkesh Patel

211,158 Aufrufe • vor 11 Monaten

this was the most important bit of the ilya interview imo. humans are not AGI, and LLMs won’t become like humans with more scaling. humans lack a huge amount of knowledge and have insanely good “continual learning” powered by extremely good reward functions (except for addiction) that guide them. and so it’s more about the learning rather than knowledge. the term AGI was coined as a response to the narrow AI which was only capable of intelligence in a specific domain (chess, game, code, etc) but General intelligence is able to learn different topics because that’s what “learning” is about. pre-training was thought to be the solution to reaching AGI because you train AI on so many different topics. but they have “overshot” that target. amazing perspective.

this was the most important bit of the ilya interview imo. humans are not AGI, and LLMs won’t become like humans with more scaling. humans lack a huge amount of knowledge and have insanely good “continual learning” powered by extremely good reward functions (except for addiction) that guide them. and so it’s more about the learning rather than knowledge. the term AGI was coined as a response to the narrow AI which was only capable of intelligence in a specific domain (chess, game, code, etc) but General intelligence is able to learn different topics because that’s what “learning” is about. pre-training was thought to be the solution to reaching AGI because you train AI on so many different topics. but they have “overshot” that target. amazing perspective.

ℏεsam

53,444 Aufrufe • vor 9 Monaten

Here's my conversation with Yann LeCun (Yann LeCun) about AI, importance of open source, limits of LLMs, why AI doomers are wrong, and the path to AGI. This was a fun and fascinating technical conversation! It's here on X in full, and is up on YouTube, Spotify, and everywhere else. Links in comment. Timestamps: 0:00 - Introduction 2:18 - Limits of LLMs 13:54 - Bilingualism and thinking 17:46 - Video prediction 25:07 - JEPA (Joint-Embedding Predictive Architecture) 28:15 - JEPA vs LLMs 37:31 - DINO and I-JEPA 38:51 - V-JEPA 44:22 - Hierarchical planning 50:40 - Autoregressive LLMs 1:06:06 - AI hallucination 1:11:30 - Reasoning in AI 1:29:02 - Reinforcement learning 1:34:10 - Woke AI 1:43:48 - Open source 1:47:26 - AI and ideology 1:49:58 - Marc Andreesen 1:57:56 - Llama 3 2:04:20 - AGI 2:08:48 - AI doomers 2:24:38 - Joscha Bach 2:28:51 - Humanoid robots 2:38:00 - Hope for the future

Here's my conversation with Yann LeCun (Yann LeCun) about AI, importance of open source, limits of LLMs, why AI doomers are wrong, and the path to AGI. This was a fun and fascinating technical conversation! It's here on X in full, and is up on YouTube, Spotify, and everywhere else. Links in comment. Timestamps: 0:00 - Introduction 2:18 - Limits of LLMs 13:54 - Bilingualism and thinking 17:46 - Video prediction 25:07 - JEPA (Joint-Embedding Predictive Architecture) 28:15 - JEPA vs LLMs 37:31 - DINO and I-JEPA 38:51 - V-JEPA 44:22 - Hierarchical planning 50:40 - Autoregressive LLMs 1:06:06 - AI hallucination 1:11:30 - Reasoning in AI 1:29:02 - Reinforcement learning 1:34:10 - Woke AI 1:43:48 - Open source 1:47:26 - AI and ideology 1:49:58 - Marc Andreesen 1:57:56 - Llama 3 2:04:20 - AGI 2:08:48 - AI doomers 2:24:38 - Joscha Bach 2:28:51 - Humanoid robots 2:38:00 - Hope for the future

Lex Fridman

1,022,504 Aufrufe • vor 2 Jahren

Episode 64 with Claynosaurz This is our dot com era. This is the future and we believe that. We have this current model that works We spoke about the past, present and future of Claynosaurz, IP tokenization, DATs It was great to have a casual sit down with the team and talk shop Timestamps: 0:01:58 Intro 0:04:38 Clayno quarterly workshop 0:05:56 Cab 🌋 new role at Claynosaurz 0:10:23 Navy seals 0:16:01 Tokenize everything 0:20:07 "Disney will buy us or we will buy them" Andrew 0:23:11 The thing we are really thinking about 0:29:20 Future of digital asset treasuries 0:34:30 GameSquare Holdings Inc. acquiring high value punks 0:37:50 Alpha 0:40:47 How is it to have DAVID HORVATH on the team 0:44:00 Assessing the Asian markets 0:48:00 Retaining top talent and the core team 0:48:50 William Liu joins the set + new position 0:56:29 Clayno's first event experience ever 1:00:10 How Plaposaur | CC 🌋 and the team went full time Clayno 1:07:34 Team member growth 1:17:02 Disney 1:20:00 Future of IP 1:30:30 "No way im saying this on air"

Episode 64 with Claynosaurz This is our dot com era. This is the future and we believe that. We have this current model that works We spoke about the past, present and future of Claynosaurz, IP tokenization, DATs It was great to have a casual sit down with the team and talk shop Timestamps: 0:01:58 Intro 0:04:38 Clayno quarterly workshop 0:05:56 Cab 🌋 new role at Claynosaurz 0:10:23 Navy seals 0:16:01 Tokenize everything 0:20:07 "Disney will buy us or we will buy them" Andrew 0:23:11 The thing we are really thinking about 0:29:20 Future of digital asset treasuries 0:34:30 GameSquare Holdings Inc. acquiring high value punks 0:37:50 Alpha 0:40:47 How is it to have DAVID HORVATH on the team 0:44:00 Assessing the Asian markets 0:48:00 Retaining top talent and the core team 0:48:50 William Liu joins the set + new position 0:56:29 Clayno's first event experience ever 1:00:10 How Plaposaur | CC 🌋 and the team went full time Clayno 1:07:34 Team member growth 1:17:02 Disney 1:20:00 Future of IP 1:30:30 "No way im saying this on air"

Andrew Forte

26,952 Aufrufe • vor 10 Monaten

New episode with former US Treasury Secretary and current OpenAI board member Larry Summers (Lawrence H. Summers). We discuss: - How he's been learning about AI. - The odds of the technology delivering a much faster economic growth regime (akin to the Industrial Revolution). - How AGI might change economic policymaking. Enjoy! Timestamps: (0:00:00) - Introduction. (0:00:46) - Larry's journey teaching himself about AI & deep learning since joining OpenAI’s board. (0:09:15) - How many hours per week has Larry been spending on OpenAI-related stuff? (0:10:16) - Which bottleneck to AI scaling does Larry think is the most underrated? (0:12:22) - Approximately what share of time do today's AI researchers spend on tasks that AI will be doing for them in five years? (0:15:01) - What explains the remarkable steadiness of US economic growth over the last 150 years? (0:19:42) - How likely is it that AI initiates a new growth regime with average growth that’s ~10x faster than today? (0:21:36) - What are the best economic arguments for believing AI won’t deliver a regime of ever-increasing growth rates? (0:25:33) - How much could AGI boost economic growth in developing countries merely by helping their policymakers make better decisions? (0:28:20) - How much better could monetary policy be if the Fed had AGI? (0:31:40) - How much would having AGI have helped US economic policymakers during the financial crisis & Great Recession? (0:36:07) - Is the CCP infiltrating and stealing the IP of major AI labs in the US and UK? (0:39:35) - At what point should AI be nationalised? (0:42:39) - How would Bill Clinton or Barack Obama be thinking about AI governance? (0:44:27) - If OpenAI restructures to a public benefit corporation, how does that change its incentives? (0:46:18) - What does Daron Acemoglu miss in his analysis of the economic impacts of AI?

New episode with former US Treasury Secretary and current OpenAI board member Larry Summers (Lawrence H. Summers). We discuss: - How he's been learning about AI. - The odds of the technology delivering a much faster economic growth regime (akin to the Industrial Revolution). - How AGI might change economic policymaking. Enjoy! Timestamps: (0:00:00) - Introduction. (0:00:46) - Larry's journey teaching himself about AI & deep learning since joining OpenAI’s board. (0:09:15) - How many hours per week has Larry been spending on OpenAI-related stuff? (0:10:16) - Which bottleneck to AI scaling does Larry think is the most underrated? (0:12:22) - Approximately what share of time do today's AI researchers spend on tasks that AI will be doing for them in five years? (0:15:01) - What explains the remarkable steadiness of US economic growth over the last 150 years? (0:19:42) - How likely is it that AI initiates a new growth regime with average growth that’s ~10x faster than today? (0:21:36) - What are the best economic arguments for believing AI won’t deliver a regime of ever-increasing growth rates? (0:25:33) - How much could AGI boost economic growth in developing countries merely by helping their policymakers make better decisions? (0:28:20) - How much better could monetary policy be if the Fed had AGI? (0:31:40) - How much would having AGI have helped US economic policymakers during the financial crisis & Great Recession? (0:36:07) - Is the CCP infiltrating and stealing the IP of major AI labs in the US and UK? (0:39:35) - At what point should AI be nationalised? (0:42:39) - How would Bill Clinton or Barack Obama be thinking about AI governance? (0:44:27) - If OpenAI restructures to a public benefit corporation, how does that change its incentives? (0:46:18) - What does Daron Acemoglu miss in his analysis of the economic impacts of AI?

Joseph Noel Walker

85,856 Aufrufe • vor 1 Jahr

NEW EPISODE: “The push and pull we have is just us. Trying to find your rhythm between the guitars, bass, and drums–it is uniquely our personalities.” - Albert Hammond Jr of The Strokes 0:00:00 Albert Hammond, Jr. of The Strokes 0:03:25 Meeting Julian & the Birth of The Strokes 0:07:22 Early Shows & the New York Scene 0:15:15 The First EP 0:34:23 The Band Members 0:39:11 Recording Is This It 0:49:44 Touring, 9/11 & SNL 0:53:57 Room on Fire & First Impressions of Earth 1:12:33 The Break, Solo Work & Rehab 1:28:22 Angles, Come Down Machine & The New Abnormal 1:53:05 A New Generation & the Current Tour

NEW EPISODE: “The push and pull we have is just us. Trying to find your rhythm between the guitars, bass, and drums–it is uniquely our personalities.” - Albert Hammond Jr of The Strokes 0:00:00 Albert Hammond, Jr. of The Strokes 0:03:25 Meeting Julian & the Birth of The Strokes 0:07:22 Early Shows & the New York Scene 0:15:15 The First EP 0:34:23 The Band Members 0:39:11 Recording Is This It 0:49:44 Touring, 9/11 & SNL 0:53:57 Room on Fire & First Impressions of Earth 1:12:33 The Break, Solo Work & Rehab 1:28:22 Angles, Come Down Machine & The New Abnormal 1:53:05 A New Generation & the Current Tour

Tetragrammaton

108,843 Aufrufe • vor 1 Monat

I had a fantastic time discussing with the learning legend Justin Skycak from Math Academy about learning math in the modern age. we've talked about his quite impressive self-learning journey (3000h of math in high school) all the way to how he hand curated the initial knowledge graph for math academy to make that process more efficient. great lively 3h discussion here are the chapters: 0:00:00 - intro: 0:02:10 - justin background 0:05:45 - 3000h math self study in high school 0:11:45 - what a day looked like for that 3000h stretch 0:16:10 - meta-learning vs pure math learning 0:21:50 - when did you get into cognitive neuro? 0:29:55 - how did the fundamental math helped in your research projects 0:43:10 - what does the math academy learning system looks like 0:47:34 - how did you guys build the 2000 topic knowledge graph 1:01:15 - would LLM be useful as an interface to that knowledge graph for the students? 1:10:46 - how does the FIRe spaced repetition algorithm works? 1:17:34 - does the same knowledge graph structure would work for physics? or other topic?: 1:34:05 - how do you understand the subject vs the curiculum 1:35:50 - is there a connection between studying math and learning a sport? 1:42:00 - do you think in math doing and teaching requires different skills? 1:56:25 - could you get understanding without automaticy? 2:05:35 - do you see any upside of confusion in learning? 2:14:11 - learning math as an adult? 2:19:20 - how to fill the motivation gap after learning the fundamental? 2:24:10 - how should teaching math for kids and adults balance fundamentals and creativity? 2:33:55 - is it ever too late to learn math seriously? 2:46:00 - mastery learning vs ultra learning 2:51:30 - top-down vs bottom-up 2:53:40 - mastery learning for domain without a structured hierarchical structure? 2:56:30 - neurodivergence / adhd for structured math learning? 3:06:20 - amateur mathematician augmented with technology will be able to contribute to research? 3:14:37 - what are you most excited about right now in term of learning enjoy!

I had a fantastic time discussing with the learning legend Justin Skycak from Math Academy about learning math in the modern age. we've talked about his quite impressive self-learning journey (3000h of math in high school) all the way to how he hand curated the initial knowledge graph for math academy to make that process more efficient. great lively 3h discussion here are the chapters: 0:00:00 - intro: 0:02:10 - justin background 0:05:45 - 3000h math self study in high school 0:11:45 - what a day looked like for that 3000h stretch 0:16:10 - meta-learning vs pure math learning 0:21:50 - when did you get into cognitive neuro? 0:29:55 - how did the fundamental math helped in your research projects 0:43:10 - what does the math academy learning system looks like 0:47:34 - how did you guys build the 2000 topic knowledge graph 1:01:15 - would LLM be useful as an interface to that knowledge graph for the students? 1:10:46 - how does the FIRe spaced repetition algorithm works? 1:17:34 - does the same knowledge graph structure would work for physics? or other topic?: 1:34:05 - how do you understand the subject vs the curiculum 1:35:50 - is there a connection between studying math and learning a sport? 1:42:00 - do you think in math doing and teaching requires different skills? 1:56:25 - could you get understanding without automaticy? 2:05:35 - do you see any upside of confusion in learning? 2:14:11 - learning math as an adult? 2:19:20 - how to fill the motivation gap after learning the fundamental? 2:24:10 - how should teaching math for kids and adults balance fundamentals and creativity? 2:33:55 - is it ever too late to learn math seriously? 2:46:00 - mastery learning vs ultra learning 2:51:30 - top-down vs bottom-up 2:53:40 - mastery learning for domain without a structured hierarchical structure? 2:56:30 - neurodivergence / adhd for structured math learning? 3:06:20 - amateur mathematician augmented with technology will be able to contribute to research? 3:14:37 - what are you most excited about right now in term of learning enjoy!

Yacine Mahdid

57,320 Aufrufe • vor 4 Monaten

.Satya Nadella on: - why he doesn’t believe in AGI but does believe in 10% economic growth - Microsoft’s new topological qubit breakthrough and gaming world models - whether Office commoditizes LLMs or the other way around Links below. Enjoy! Timestamps 0:00:00 - Intro 0:05:48 - AI won't be winner-take-all 0:16:02 - World economy growing by 10% 0:22:23 - Decreasing price of intelligence 0:31:03 - Microsoft's Quantum breakthrough 0:43:35 - Microsoft's gaming world model 0:50:35 - Legal barriers to AI 0:56:30 - Getting AGI safety right 1:05:43 - 34 years at Microsoft 1:11:31 - Does Satya Nadella believe in AGI?

.Satya Nadella on: - why he doesn’t believe in AGI but does believe in 10% economic growth - Microsoft’s new topological qubit breakthrough and gaming world models - whether Office commoditizes LLMs or the other way around Links below. Enjoy! Timestamps 0:00:00 - Intro 0:05:48 - AI won't be winner-take-all 0:16:02 - World economy growing by 10% 0:22:23 - Decreasing price of intelligence 0:31:03 - Microsoft's Quantum breakthrough 0:43:35 - Microsoft's gaming world model 0:50:35 - Legal barriers to AI 0:56:30 - Getting AGI safety right 1:05:43 - 34 years at Microsoft 1:11:31 - Does Satya Nadella believe in AGI?

Dwarkesh Patel

2,264,372 Aufrufe • vor 1 Jahr

Today we release my favorite episode of Training Data yet: the great Rich Sutton. Richard Sutton wrote the textbook, wrote The Bitter Lesson (and many other on-point essays like "Self-Verification, The Key to AI"), and trained a mafia of talented students who went on to change the AI landscape forever including David Silver, inventor of built AlphaGo. Khurram Javed was Rich's PhD student at Alberta and wrote The Big World Hypothesis. They just left academia to start Oak Lab Their core argument: (1) The Bitter Lesson: the world is massively more complex than any model of it, so anything trained on human-curated data has a ceiling (2) Continual Learning: intelligence is continual by definition, and today's models stop learning the moment they ship. The conversation covers: — what The Bitter Lesson actually says, and what people get wrong — why synthetic data is "just a big mistake," and the Big World Hypothesis behind it — how LLMs are both a positive and a negative example of his own essay — why no animal learns by supervised learning, and what squirrels can do that we can't — the cure for catastrophic forgetting: per-weight step sizes and continual backprop — why the biggest labs can't take a path where performance gets worse before it gets better — a trillion parameters on 20 watts, and the Moore's Law math that makes it plausible — why the endpoint isn't one mind but one design, running as many minds It was both a fun generative idea- and debate-filled conversation, and a surprisingly human one too. Rich, thank you for beating cancer and changing the trajectory of AI. 💙 00:00 Introduction 02:10 An AI winter, a cancer diagnosis, and the move to Alberta 07:07 Writing "The Bitter Lesson," and what people get wrong 09:53 Are LLMs a positive or a negative example of it? 11:03 Synthetic data is "just a big mistake," and the Big World Hypothesis 18:01 AlphaGo, human priors, and why prior knowledge and learning should be friends 22:37 "Their weights never change": do LLM assistants actually learn? 26:09 Babies, squirrels, and why no animal learns by supervised learning 32:02 Rockets, imagination, and where paradigm shifts come from 36:42 The Alberta Plan and its 12 steps 38:53 Catastrophic forgetting and the cure 43:43 Oak's biggest ambition: a self-maintaining mind 47:56 Why the big labs are stuck in a local minimum 49:13 If everything goes right: LLMs, many minds, and hiring The man who pioneered reinforcement learning thinks the rest of the field is weird, and lays it all out in today's episode. Together w/ Alfred Lin Sequoia Capital

Today we release my favorite episode of Training Data yet: the great Rich Sutton. Richard Sutton wrote the textbook, wrote The Bitter Lesson (and many other on-point essays like "Self-Verification, The Key to AI"), and trained a mafia of talented students who went on to change the AI landscape forever including David Silver, inventor of built AlphaGo. Khurram Javed was Rich's PhD student at Alberta and wrote The Big World Hypothesis. They just left academia to start Oak Lab Their core argument: (1) The Bitter Lesson: the world is massively more complex than any model of it, so anything trained on human-curated data has a ceiling (2) Continual Learning: intelligence is continual by definition, and today's models stop learning the moment they ship. The conversation covers: — what The Bitter Lesson actually says, and what people get wrong — why synthetic data is "just a big mistake," and the Big World Hypothesis behind it — how LLMs are both a positive and a negative example of his own essay — why no animal learns by supervised learning, and what squirrels can do that we can't — the cure for catastrophic forgetting: per-weight step sizes and continual backprop — why the biggest labs can't take a path where performance gets worse before it gets better — a trillion parameters on 20 watts, and the Moore's Law math that makes it plausible — why the endpoint isn't one mind but one design, running as many minds It was both a fun generative idea- and debate-filled conversation, and a surprisingly human one too. Rich, thank you for beating cancer and changing the trajectory of AI. 💙 00:00 Introduction 02:10 An AI winter, a cancer diagnosis, and the move to Alberta 07:07 Writing "The Bitter Lesson," and what people get wrong 09:53 Are LLMs a positive or a negative example of it? 11:03 Synthetic data is "just a big mistake," and the Big World Hypothesis 18:01 AlphaGo, human priors, and why prior knowledge and learning should be friends 22:37 "Their weights never change": do LLM assistants actually learn? 26:09 Babies, squirrels, and why no animal learns by supervised learning 32:02 Rockets, imagination, and where paradigm shifts come from 36:42 The Alberta Plan and its 12 steps 38:53 Catastrophic forgetting and the cure 43:43 Oak's biggest ambition: a self-maintaining mind 47:56 Why the big labs are stuck in a local minimum 49:13 If everything goes right: LLMs, many minds, and hiring The man who pioneered reinforcement learning thinks the rest of the field is weird, and lays it all out in today's episode. Together w/ Alfred Lin Sequoia Capital

Sonya Huang 🐥

111,326 Aufrufe • vor 12 Tagen

This morning at NeurIPS, Rich Sutton reminded us that we need continual learning to reach AGI. This afternoon, Ali Behrouz presented a Google poster paper, Nested Learning, which provides new ideas on the path to continual learning. I recorded the 40 minute talk as it might be useful for some researchers in the audience. For the rest of us, I subscribe to Andrej Karpathy's suspicion that it will take a 5-10 papers like this to move us to AGI from where we are now, just like it took about 10 papers to move from 2012's AlexNet to ChatGPT. At the very end, I ask Ali how far along to continual learning this represents. Full paper link below, as well as a YouTube link. ps. sorry about the first 2 minutes of bad audio since there were 2 idiots standing beside me have a conversation right in front of this presenter in a rather packed poster presentation. Honestly, tamp down your egos guys and show come common courtesy!

This morning at NeurIPS, Rich Sutton reminded us that we need continual learning to reach AGI. This afternoon, Ali Behrouz presented a Google poster paper, Nested Learning, which provides new ideas on the path to continual learning. I recorded the 40 minute talk as it might be useful for some researchers in the audience. For the rest of us, I subscribe to Andrej Karpathy's suspicion that it will take a 5-10 papers like this to move us to AGI from where we are now, just like it took about 10 papers to move from 2012's AlexNet to ChatGPT. At the very end, I ask Ali how far along to continual learning this represents. Full paper link below, as well as a YouTube link. ps. sorry about the first 2 minutes of bad audio since there were 2 idiots standing beside me have a conversation right in front of this presenter in a rather packed poster presentation. Honestly, tamp down your egos guys and show come common courtesy!

Phil Trubey

235,587 Aufrufe • vor 9 Monaten

The Delta of Intelligence is Human Data! Curtis G. Northcutt is Director of AI Research at Handshake AI. In this episode, Curtis shares his unique perspectives on human data, RL envs, four quadrants of the AI market, journey of Handshake and much more. 0:00:00 - INTRO 0:02:08 - The Snowman Effect: From Rural Kentucky to 7 Years at MIT 0:09:50 - Confident Learning, Two Systems of Intelligence, Classical ML Has to Prevail 0:20:37 - Origin of CleanLab and Handshake acquisition 0:28:00 - How does GPT-5 become GPT-6? 0:31:20 - The Fastest growing Data Lab: What makes Handshake different? 0:35:32 - Difference between Good Data and Bad Data 0:38:17 - New kind of Data, IA and AI 0:42:04 - Scaling Coding Benchmarks and Efficiency in Long Horizon Tasks 0:49:12 - On Pre-Training, Misconception around Human Data 0:57:13 - Open Question: Taste and Personality, Quick Fire 01:05:15 - Advice to 20yo : Skill and Obsession

The Delta of Intelligence is Human Data! Curtis G. Northcutt is Director of AI Research at Handshake AI. In this episode, Curtis shares his unique perspectives on human data, RL envs, four quadrants of the AI market, journey of Handshake and much more. 0:00:00 - INTRO 0:02:08 - The Snowman Effect: From Rural Kentucky to 7 Years at MIT 0:09:50 - Confident Learning, Two Systems of Intelligence, Classical ML Has to Prevail 0:20:37 - Origin of CleanLab and Handshake acquisition 0:28:00 - How does GPT-5 become GPT-6? 0:31:20 - The Fastest growing Data Lab: What makes Handshake different? 0:35:32 - Difference between Good Data and Bad Data 0:38:17 - New kind of Data, IA and AI 0:42:04 - Scaling Coding Benchmarks and Efficiency in Long Horizon Tasks 0:49:12 - On Pre-Training, Misconception around Human Data 0:57:13 - Open Question: Taste and Personality, Quick Fire 01:05:15 - Advice to 20yo : Skill and Obsession

himanshu

22,264 Aufrufe • vor 3 Monaten

We are excited to share that Logan Kilpatrick joined us on The Bench to discuss Google's new Gemini 3.5 Flash: why it's deliberately more persistent and capable than previous Flash models, how it hit #1 on our FinanceAgent Benchmark taking 82 steps where competitors stopped at 13, and what justifies the price increase. We also get into why AI benchmarks need a paradigm shift, the trade-off of building everything vs staying focused, the Pope, and why Omni might kill the Subway Surfers content era. 0:11:00 – Flash is being rebased for the agent era, not just a cheaper model anymore 0:14:03 – Persistence by design: 82 tool calls vs competitors' 13 0:17:52 – Why pricing went up and how Google thinks about value per token 0:22:55 – Coding performance: from 20th to 10th place in one generation 0:28:28 – Why benchmarks have historically been misleading and what the new era of evaluation looks like 0:29:28 Logan on why Google has the best researchers in the world 0:36:16 – The cost of being Google 0:39:07 – The Pope’s encyclical on AI and whether most people see frontier intelligence as a good thing 0:51:12 – Why Omni is the thing that recently clicked

We are excited to share that Logan Kilpatrick joined us on The Bench to discuss Google's new Gemini 3.5 Flash: why it's deliberately more persistent and capable than previous Flash models, how it hit #1 on our FinanceAgent Benchmark taking 82 steps where competitors stopped at 13, and what justifies the price increase. We also get into why AI benchmarks need a paradigm shift, the trade-off of building everything vs staying focused, the Pope, and why Omni might kill the Subway Surfers content era. 0:11:00 – Flash is being rebased for the agent era, not just a cheaper model anymore 0:14:03 – Persistence by design: 82 tool calls vs competitors' 13 0:17:52 – Why pricing went up and how Google thinks about value per token 0:22:55 – Coding performance: from 20th to 10th place in one generation 0:28:28 – Why benchmarks have historically been misleading and what the new era of evaluation looks like 0:29:28 Logan on why Google has the best researchers in the world 0:36:16 – The cost of being Google 0:39:07 – The Pope’s encyclical on AI and whether most people see frontier intelligence as a good thing 0:51:12 – Why Omni is the thing that recently clicked

Vals AI

16,486 Aufrufe • vor 3 Monaten