Загрузка видео...

Не удалось загрузить видео

На главную

.Richard Sutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled. My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning. And if we have continual learning, we don't need a special training phase - the agent just learns on-the-fly - like all...

3,080,205 просмотров • 9 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

"Projects like the New Deal, the Apollo program pale in comparison to what we're doing right now." 🆕 Greg Brockman (Greg Brockman) joins us to talk GPT-5, GPT-OSS, and what's next on OpenAI's road to crystallizing all of human intelligence! “Energy turns into compute, turns into intelligence… crystallizing compute into potential energy you can release again and again.” 0:00:04 - Introductions 0:01:04 - The Evolution of Reasoning at OpenAI 0:04:01 - Online vs Offline Learning in Language Models 0:06:44 - Sample Efficiency and Human Curation in Reinforcement Learning 0:08:16 - Scaling Compute and Supercritical Learning 0:13:21 - Wall clock time limitations in RL and real-world interactions 0:16:34 - Experience with ARC Institute and DNA neural networks 0:19:33 - Defining the GPT-5 Era 0:22:46 - Evaluating Model Intelligence and Task Difficulty 0:25:06 - Practical Advice for Developers Using GPT-5 0:31:48 - Model Specs 0:37:21 - Challenges in RL Preferences (e.g., try/catch) 0:39:13 - Model Routing and Hybrid Architectures in GPT-5 0:43:58 - GPT-5 pricing and compute efficiency improvements 0:46:04 - Self-Improving Coding Agents and Tool Usage 0:49:11 - On-Device Models and Local vs Remote Agent Systems 0:51:34 - Engineering at OpenAI and Leveraging LLMs 0:54:16 - Structuring Codebases and Teams for AI Optimization 0:55:27 - The Value of Engineers in the Age of AGI 0:58:42 - Current state of AI research and lab diversity 1:01:11 - OpenAI’s Prioritization and Focus Areas 1:03:05 - Advice for Founders - It's Not Too Late 1:04:20 - Future outlook and closing thoughts 1:04:33 - Time Capsule to 2045 - Future of Compute and Abundance 1:07:07 - Time Capsule to 2005 - More Problems Will Emerge

Latent.Space

305,090 просмотров • 11 месяцев назад

New episode with former US Treasury Secretary and current OpenAI board member Larry Summers (Lawrence H. Summers). We discuss: - How he's been learning about AI. - The odds of the technology delivering a much faster economic growth regime (akin to the Industrial Revolution). - How AGI might change economic policymaking. Enjoy! Timestamps: (0:00:00) - Introduction. (0:00:46) - Larry's journey teaching himself about AI & deep learning since joining OpenAI’s board. (0:09:15) - How many hours per week has Larry been spending on OpenAI-related stuff? (0:10:16) - Which bottleneck to AI scaling does Larry think is the most underrated? (0:12:22) - Approximately what share of time do today's AI researchers spend on tasks that AI will be doing for them in five years? (0:15:01) - What explains the remarkable steadiness of US economic growth over the last 150 years? (0:19:42) - How likely is it that AI initiates a new growth regime with average growth that’s ~10x faster than today? (0:21:36) - What are the best economic arguments for believing AI won’t deliver a regime of ever-increasing growth rates? (0:25:33) - How much could AGI boost economic growth in developing countries merely by helping their policymakers make better decisions? (0:28:20) - How much better could monetary policy be if the Fed had AGI? (0:31:40) - How much would having AGI have helped US economic policymakers during the financial crisis & Great Recession? (0:36:07) - Is the CCP infiltrating and stealing the IP of major AI labs in the US and UK? (0:39:35) - At what point should AI be nationalised? (0:42:39) - How would Bill Clinton or Barack Obama be thinking about AI governance? (0:44:27) - If OpenAI restructures to a public benefit corporation, how does that change its incentives? (0:46:18) - What does Daron Acemoglu miss in his analysis of the economic impacts of AI?

Joseph Noel Walker

85,856 просмотров • 1 год назад

I had a fantastic time discussing with the learning legend Justin Skycak from Math Academy about learning math in the modern age. we've talked about his quite impressive self-learning journey (3000h of math in high school) all the way to how he hand curated the initial knowledge graph for math academy to make that process more efficient. great lively 3h discussion here are the chapters: 0:00:00 - intro: 0:02:10 - justin background 0:05:45 - 3000h math self study in high school 0:11:45 - what a day looked like for that 3000h stretch 0:16:10 - meta-learning vs pure math learning 0:21:50 - when did you get into cognitive neuro? 0:29:55 - how did the fundamental math helped in your research projects 0:43:10 - what does the math academy learning system looks like 0:47:34 - how did you guys build the 2000 topic knowledge graph 1:01:15 - would LLM be useful as an interface to that knowledge graph for the students? 1:10:46 - how does the FIRe spaced repetition algorithm works? 1:17:34 - does the same knowledge graph structure would work for physics? or other topic?: 1:34:05 - how do you understand the subject vs the curiculum 1:35:50 - is there a connection between studying math and learning a sport? 1:42:00 - do you think in math doing and teaching requires different skills? 1:56:25 - could you get understanding without automaticy? 2:05:35 - do you see any upside of confusion in learning? 2:14:11 - learning math as an adult? 2:19:20 - how to fill the motivation gap after learning the fundamental? 2:24:10 - how should teaching math for kids and adults balance fundamentals and creativity? 2:33:55 - is it ever too late to learn math seriously? 2:46:00 - mastery learning vs ultra learning 2:51:30 - top-down vs bottom-up 2:53:40 - mastery learning for domain without a structured hierarchical structure? 2:56:30 - neurodivergence / adhd for structured math learning? 3:06:20 - amateur mathematician augmented with technology will be able to contribute to research? 3:14:37 - what are you most excited about right now in term of learning enjoy!

Yacine Mahdid

57,320 просмотров • 3 месяцев назад

The most interesting part for me is where Andrej Karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.” A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer. > “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.” So what do humans do instead? > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” > “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.” Why can’t we just add this training to LLMs today? > “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.” > “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.” > “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.” How do humans get around model collapse? > “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.” In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by Erik Hoel. I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization? > “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”

Dwarkesh Patel

1,050,747 просмотров • 8 месяцев назад