Loading video...

Video Failed to Load

Go Home

Everybody is talking about recursive self-improvement (RSI) and meta learning. Here is my old 2020 talk about this [1]. It has aged well. Example: humans still define the starts & ends of trials of many modern meta learners. My RSI systems since 1994 LEARN to (re)define them [2]! [1]...

SchmidhuberAI's profile picture

Jürgen Schmidhuber

209,921 subscribers

223,861 views • 4 months ago •via X (Twitter)

EducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

New Paper! Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents A longstanding goal of AI research has been the creation of AI that can learn indefinitely. One path toward that goal is an AI that improves itself by rewriting its own code, including any code responsible for learning. That idea, known as a Gödel Machine, proposed by Jürgen Schmidhuber over two decades ago, is a hypothetical self-improving AI. It optimally solves problems by recursively rewriting its own code when it can mathematically prove a better strategy, making it a key concept in meta-learning or “learning to learn.” While the theoretical Gödel Machine promised provably beneficial self-modifications, its realization relied on an impractical assumption: that the AI could mathematically prove that a proposed change in its own code would yield a net improvement before adopting it. Sakana AI, in collaboration with Jeff Clune’s lab at UBC, proposes something more feasible: a system that harnesses the principles of open-ended algorithms like Darwinian evolution to search for improvements that empirically improve performance. We call the result the Darwin Gödel Machine. DGMs leverage foundation models to propose code improvements, and use recent innovations in open-ended algorithms to search for a growing library of diverse, high-quality AI agents. Applied to practical tasks, we implemented Darwin Gödel Machine as a self-improving coding agent that rewrites its own code to improve performance on programming tasks. It creates various self-improvements, such as a patch validation step, better file viewing, enhanced editing tools, generating and ranking multiple solutions to choose the best one, and adding a history of what has been tried before (and why it failed) when making new changes (see the attached video). We believe that Darwin Gödel Machines represent a concrete step towards AI systems that can autonomously gather their own stepping stones to learn and innovate forever!
0:27

New Paper! Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents A longstanding goal of AI research has been the creation of AI that can learn indefinitely. One path toward that goal is an AI that improves itself by rewriting its own code, including any code responsible for learning. That idea, known as a Gödel Machine, proposed by Jürgen Schmidhuber over two decades ago, is a hypothetical self-improving AI. It optimally solves problems by recursively rewriting its own code when it can mathematically prove a better strategy, making it a key concept in meta-learning or “learning to learn.” While the theoretical Gödel Machine promised provably beneficial self-modifications, its realization relied on an impractical assumption: that the AI could mathematically prove that a proposed change in its own code would yield a net improvement before adopting it. Sakana AI, in collaboration with Jeff Clune’s lab at UBC, proposes something more feasible: a system that harnesses the principles of open-ended algorithms like Darwinian evolution to search for improvements that empirically improve performance. We call the result the Darwin Gödel Machine. DGMs leverage foundation models to propose code improvements, and use recent innovations in open-ended algorithms to search for a growing library of diverse, high-quality AI agents. Applied to practical tasks, we implemented Darwin Gödel Machine as a self-improving coding agent that rewrites its own code to improve performance on programming tasks. It creates various self-improvements, such as a patch validation step, better file viewing, enhanced editing tools, generating and ranking multiple solutions to choose the best one, and adding a history of what has been tried before (and why it failed) when making new changes (see the attached video). We believe that Darwin Gödel Machines represent a concrete step towards AI systems that can autonomously gather their own stepping stones to learn and innovate forever!

hardmaru

104,854 views • 1 year ago

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.
1:05

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.

Khurram Javed

52,110 views • 8 months ago

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.
2:35

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

AI at Meta

453,260 views • 1 year ago

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)
1:07

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)

Rohan Paul

440,176 views • 13 days ago

I had a fantastic time discussing with the learning legend Justin Skycak from Math Academy about learning math in the modern age. we've talked about his quite impressive self-learning journey (3000h of math in high school) all the way to how he hand curated the initial knowledge graph for math academy to make that process more efficient. great lively 3h discussion here are the chapters: 0:00:00 - intro: 0:02:10 - justin background 0:05:45 - 3000h math self study in high school 0:11:45 - what a day looked like for that 3000h stretch 0:16:10 - meta-learning vs pure math learning 0:21:50 - when did you get into cognitive neuro? 0:29:55 - how did the fundamental math helped in your research projects 0:43:10 - what does the math academy learning system looks like 0:47:34 - how did you guys build the 2000 topic knowledge graph 1:01:15 - would LLM be useful as an interface to that knowledge graph for the students? 1:10:46 - how does the FIRe spaced repetition algorithm works? 1:17:34 - does the same knowledge graph structure would work for physics? or other topic?: 1:34:05 - how do you understand the subject vs the curiculum 1:35:50 - is there a connection between studying math and learning a sport? 1:42:00 - do you think in math doing and teaching requires different skills? 1:56:25 - could you get understanding without automaticy? 2:05:35 - do you see any upside of confusion in learning? 2:14:11 - learning math as an adult? 2:19:20 - how to fill the motivation gap after learning the fundamental? 2:24:10 - how should teaching math for kids and adults balance fundamentals and creativity? 2:33:55 - is it ever too late to learn math seriously? 2:46:00 - mastery learning vs ultra learning 2:51:30 - top-down vs bottom-up 2:53:40 - mastery learning for domain without a structured hierarchical structure? 2:56:30 - neurodivergence / adhd for structured math learning? 3:06:20 - amateur mathematician augmented with technology will be able to contribute to research? 3:14:37 - what are you most excited about right now in term of learning enjoy!
3:18:00

I had a fantastic time discussing with the learning legend Justin Skycak from Math Academy about learning math in the modern age. we've talked about his quite impressive self-learning journey (3000h of math in high school) all the way to how he hand curated the initial knowledge graph for math academy to make that process more efficient. great lively 3h discussion here are the chapters: 0:00:00 - intro: 0:02:10 - justin background 0:05:45 - 3000h math self study in high school 0:11:45 - what a day looked like for that 3000h stretch 0:16:10 - meta-learning vs pure math learning 0:21:50 - when did you get into cognitive neuro? 0:29:55 - how did the fundamental math helped in your research projects 0:43:10 - what does the math academy learning system looks like 0:47:34 - how did you guys build the 2000 topic knowledge graph 1:01:15 - would LLM be useful as an interface to that knowledge graph for the students? 1:10:46 - how does the FIRe spaced repetition algorithm works? 1:17:34 - does the same knowledge graph structure would work for physics? or other topic?: 1:34:05 - how do you understand the subject vs the curiculum 1:35:50 - is there a connection between studying math and learning a sport? 1:42:00 - do you think in math doing and teaching requires different skills? 1:56:25 - could you get understanding without automaticy? 2:05:35 - do you see any upside of confusion in learning? 2:14:11 - learning math as an adult? 2:19:20 - how to fill the motivation gap after learning the fundamental? 2:24:10 - how should teaching math for kids and adults balance fundamentals and creativity? 2:33:55 - is it ever too late to learn math seriously? 2:46:00 - mastery learning vs ultra learning 2:51:30 - top-down vs bottom-up 2:53:40 - mastery learning for domain without a structured hierarchical structure? 2:56:30 - neurodivergence / adhd for structured math learning? 3:06:20 - amateur mathematician augmented with technology will be able to contribute to research? 3:14:37 - what are you most excited about right now in term of learning enjoy!

Yacine Mahdid

57,320 views • 3 months ago

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹
1:31:19

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹

Yacine Mahdid

13,023 views • 3 months ago

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.
2:48

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.

Skild AI

383,061 views • 1 year ago

🚨Ray-Ban & Oakley Meta Glasses update 🚨 -Faster camera Our engineers are cracked and found a way to make media capture latency even faster with no compromises. You're welcome. - Calendar You can now connect your Google and Outlook calendars to Meta AI to receive event notifications, create personal events, and find important calendar information just by asking Meta AI. For example, say “Hey Meta, what time is my first meeting tomorrow morning?” or “Hey Meta, add yoga to my calendar tomorrow at 9 AM.” - Audible voice search expanded availability You can now use voice commands (e.g., "Hey Meta, play my audiobook") to control Audible in all English-speaking regions where both the Audible and Meta AI apps are available. - Creation location for reminders If you've turned on location services for the Meta AI app, the creation location will be saved when you create a reminder. This allows you to see the location where a reminder was created in the Meta AI app. You can also ask Meta AI about it. - Finally, in recent months, we've began to run a lot more A/B tests and experiments, so you may be seeing some other nuggets. Will shout them out when we learn and validate these on a wider scale. If you haven't already, consider opting into our Early Access Program (EAP) in Settings. As you can guess, many more cool announcements coming next month at Connect, so enjoy this POV footage of me practicing with my little QB and stay tuned!
0:38

🚨Ray-Ban & Oakley Meta Glasses update 🚨 -Faster camera Our engineers are cracked and found a way to make media capture latency even faster with no compromises. You're welcome. - Calendar You can now connect your Google and Outlook calendars to Meta AI to receive event notifications, create personal events, and find important calendar information just by asking Meta AI. For example, say “Hey Meta, what time is my first meeting tomorrow morning?” or “Hey Meta, add yoga to my calendar tomorrow at 9 AM.” - Audible voice search expanded availability You can now use voice commands (e.g., "Hey Meta, play my audiobook") to control Audible in all English-speaking regions where both the Audible and Meta AI apps are available. - Creation location for reminders If you've turned on location services for the Meta AI app, the creation location will be saved when you create a reminder. This allows you to see the location where a reminder was created in the Meta AI app. You can also ask Meta AI about it. - Finally, in recent months, we've began to run a lot more A/B tests and experiments, so you may be seeing some other nuggets. Will shout them out when we learn and validate these on a wider scale. If you haven't already, consider opting into our Early Access Program (EAP) in Settings. As you can guess, many more cool announcements coming next month at Connect, so enjoy this POV footage of me practicing with my little QB and stay tuned!

David Woodland

53,750 views • 11 months ago

Today I’m announcing a major multi-million dollar defamation lawsuit against Meta, the owner of Facebook & Instagram. The case is WILD and has implications for ALL OF US. On top of falsely calling me a criminal, Meta suggested my kids be taken from me. Here’s a summary of events: This all started with Meta’s AI falsely claiming that I was charged with a crime from January 6th but… I wasn’t even in DC that day (I was in TN) and I’ve never been charged with a crime IN MY LIFE. We found this out in August of 2024 when I was exposing woke policies at Harley Davidson. One dealership was unhappy with me and they posted a screenshot from Meta’s AI in an effort to attack me. This screenshot was filled with lies. I couldn’t believe it was real so I checked myself. It was even worse when I checked. From that day I’ve faced a steady stream of false accusations that are deeply damaging to my character and the safety of my family. This sounds bad, right? It gets MUCH worse. Meta was notified LAST YEAR by my lawyers, yet the defamation continues today. Some lies Meta spread about me: • Meta’s AI claims that I’ve appeared on Nick Fuentes show, that I’ve spoken at his rallies, and that I’ve supported him. I’ve never met him and this is all false. • Meta’s AI claims I’ve engaged in Holocaust denial. I’ve NEVER denied the Holocaust. • Meta’s AI tells advertisers NOT to advertise with me because of the lies it invented. • Meta tells employers NOT to hire me because of the lies it invented. • Meta suggested that MY KIDS BE TAKEN FROM ME because it would be better for them to be raised by someone more friendly to DEI and transgenderism. • Meta’s ironically claimed that I’ve been sued for DEFAMATION and EMOTIONAL DISTRESS. I’ve never been sued for either. I’ve tried to fix this privately since last year. Instead of fixing this and instituting safeguards, Meta has given us the runaround. Meta later BLACKLISTED my name from being searched (insane) but it didn’t end the defamation because Meta includes my name in news stories. You can then ask for more info about me. If you do that, Meta goes back to defaming me. In fact, the lies this week are the worst yet! Meta’s AI admits that a false accusation over J6 is extremely harmful to whoever is accused. It even agrees that a court is LIKELY to rule that this was defamation with ACTUAL MALICE. With my lawsuit today, I intend to MAKE Mark Zuckerberg Meta solve this problem. Why is this so important? While I’m the target today, a candidate you like could be the next target, and lies from Meta’s AI could flip votes that decide the election. YOU could be the next target too. That’s why I’m taking on this David vs. Goliath fight. For me, my honor, my family, for our elections, and FOR YOU! If you want to help me fight ALL the battles that I’m fighting, you can help support my team at Timecodes: (0:00) Intro (0:40) Announcing Meta Lawsuit (2:01) Video of Meta’s AI Defaming Me (03:24) Meta’s AI Tells People Not To Hire or Advertise With Me (04:30) How This All Started (05:35) Trying Meta’s AI Myself — Meta Claims I Was Charged With A Crime (06:06) Meta Says I Support Nick Fuentes (06:45) Meta Says I’m A Holocaust Denier (07:07) Meta’s AI Admits Courts Will Likely Find What They Did To Be Malicious (08:08) The Threat To Our Elections (09:11) More Defamation (09:27) Were Meta’s Lies Used In A Corporate Intelligence Report? (10:36) Meta’s AI Says They Need To Apologize And Institute Safeguards (11:15) Meta Admits Their Lies Have Damaged My Reputation (11:30) Our Legal Communication since 2024 (12:28) Meta Blacklists My Name + How It Lies (14:08) Meta Suggests Authorities Take My Kids Away Because I Pose A "Threat To Their Wellbeing" (16:00) Meta Asks Me For PR Help 😳 (16:31) The Consequences Of Lying AI (17:39) Threats and An Arrest Of A Man Who Wanted To Kill Me (18:30) Closing Argument (19:07) Introducing My New Show
19:58

Today I’m announcing a major multi-million dollar defamation lawsuit against Meta, the owner of Facebook & Instagram. The case is WILD and has implications for ALL OF US. On top of falsely calling me a criminal, Meta suggested my kids be taken from me. Here’s a summary of events: This all started with Meta’s AI falsely claiming that I was charged with a crime from January 6th but… I wasn’t even in DC that day (I was in TN) and I’ve never been charged with a crime IN MY LIFE. We found this out in August of 2024 when I was exposing woke policies at Harley Davidson. One dealership was unhappy with me and they posted a screenshot from Meta’s AI in an effort to attack me. This screenshot was filled with lies. I couldn’t believe it was real so I checked myself. It was even worse when I checked. From that day I’ve faced a steady stream of false accusations that are deeply damaging to my character and the safety of my family. This sounds bad, right? It gets MUCH worse. Meta was notified LAST YEAR by my lawyers, yet the defamation continues today. Some lies Meta spread about me: • Meta’s AI claims that I’ve appeared on Nick Fuentes show, that I’ve spoken at his rallies, and that I’ve supported him. I’ve never met him and this is all false. • Meta’s AI claims I’ve engaged in Holocaust denial. I’ve NEVER denied the Holocaust. • Meta’s AI tells advertisers NOT to advertise with me because of the lies it invented. • Meta tells employers NOT to hire me because of the lies it invented. • Meta suggested that MY KIDS BE TAKEN FROM ME because it would be better for them to be raised by someone more friendly to DEI and transgenderism. • Meta’s ironically claimed that I’ve been sued for DEFAMATION and EMOTIONAL DISTRESS. I’ve never been sued for either. I’ve tried to fix this privately since last year. Instead of fixing this and instituting safeguards, Meta has given us the runaround. Meta later BLACKLISTED my name from being searched (insane) but it didn’t end the defamation because Meta includes my name in news stories. You can then ask for more info about me. If you do that, Meta goes back to defaming me. In fact, the lies this week are the worst yet! Meta’s AI admits that a false accusation over J6 is extremely harmful to whoever is accused. It even agrees that a court is LIKELY to rule that this was defamation with ACTUAL MALICE. With my lawsuit today, I intend to MAKE Mark Zuckerberg Meta solve this problem. Why is this so important? While I’m the target today, a candidate you like could be the next target, and lies from Meta’s AI could flip votes that decide the election. YOU could be the next target too. That’s why I’m taking on this David vs. Goliath fight. For me, my honor, my family, for our elections, and FOR YOU! If you want to help me fight ALL the battles that I’m fighting, you can help support my team at Timecodes: (0:00) Intro (0:40) Announcing Meta Lawsuit (2:01) Video of Meta’s AI Defaming Me (03:24) Meta’s AI Tells People Not To Hire or Advertise With Me (04:30) How This All Started (05:35) Trying Meta’s AI Myself — Meta Claims I Was Charged With A Crime (06:06) Meta Says I Support Nick Fuentes (06:45) Meta Says I’m A Holocaust Denier (07:07) Meta’s AI Admits Courts Will Likely Find What They Did To Be Malicious (08:08) The Threat To Our Elections (09:11) More Defamation (09:27) Were Meta’s Lies Used In A Corporate Intelligence Report? (10:36) Meta’s AI Says They Need To Apologize And Institute Safeguards (11:15) Meta Admits Their Lies Have Damaged My Reputation (11:30) Our Legal Communication since 2024 (12:28) Meta Blacklists My Name + How It Lies (14:08) Meta Suggests Authorities Take My Kids Away Because I Pose A "Threat To Their Wellbeing" (16:00) Meta Asks Me For PR Help 😳 (16:31) The Consequences Of Lying AI (17:39) Threats and An Arrest Of A Man Who Wanted To Kill Me (18:30) Closing Argument (19:07) Introducing My New Show

Robby Starbuck

3,720,236 views • 1 year ago

AGI? One day, but not yet. The only AI that works well right now is the one behind the screen [12-17]. But passing the Turing Test [9] behind a screen is easy compared to Real AI for real robots in the real world. No current AI-driven robot could be certified as a plumber [13-17]. Hence, the Turing Test isn't a good measure of intelligence (and neither is IQ). And AGI without mastery of the physical world is no AGI. That’s why I created the TUM CogBotLab for learning robots in 2004 [5], co-founded a company for AI in the physical world in 2014 [6], and had teams at TUM, IDSIA, and now KAUST work towards baby robots [4,10-11,18]. Such soft robots don't just slavishly imitate humans and they don't work by just downloading the web like LLMs/VLMs. No. Instead, they exploit the principles of Artificial Curiosity to improve their neural World Models (two terms I used back in 1990 [1-4]). These robots work with lots of sensors, but only weak actuators, such that they cannot easily harm themselves [18] when they collect useful data by devising and running their own self-invented experiments. Remarkably, since the 1970s, many have made fun of my old goal to build a self-improving AGI smarter than myself and then retire. Recently, however, many have finally started to take this seriously, and now some of them are suddenly TOO optimistic. These people are often blissfully unaware of the remaining challenges we have to solve to achieve Real AI. My 2024 TED talk [15] summarises some of that. REFERENCES (easy to find on the web): [1] J. Schmidhuber. Making the world differentiable: On using fully recurrent self-supervised neural networks (NNs) for dynamic reinforcement learning and planning in non-stationary environments. TR FKI-126-90, TUM, Feb 1990, revised Nov 1990. This paper also introduced artificial curiosity and intrinsic motivation through generative adversarial networks where a generator NN is fighting a predictor NN in a minimax game. [2] J. S. A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer and S. W. Wilson, editors, Proc. of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pages 222-227. MIT Press/Bradford Books, 1991. Based on [1]. [3] J.S. AI Blog (2020). 1990: Planning & Reinforcement Learning with Recurrent World Models and Artificial Curiosity. Summarising aspects of [1][2] and lots of later papers including [7][8]. [4] J.S. AI Blog (2021): Artificial Curiosity & Creativity Since 1990. Summarising aspects of [1][2] and lots of later papers including [7][8]. [5] J.S. TU Munich CogBotLab for learning robots (2004-2009) [6] NNAISENSE, founded in 2014, for AI in the physical world [7] J.S. (2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning (RL) Controllers and Recurrent Neural World Models. arXiv 1210.0118. Sec. 5.3 describes an RL prompt engineer which learns to query its model for abstract reasoning and planning and decision making. Today this is called "chain of thought." [8] J.S. (2018). One Big Net For Everything. arXiv 1802.08864. See also patent US11853886B2 and my DeepSeek tweet: DeepSeek uses elements of the 2015 reinforcement learning prompt engineer [7] and its 2018 refinement [8] which collapses the RL machine and world model of [7] into a single net. This uses my neural net distillation procedure of 1991: a distilled chain of thought system. [9] J.S. Turing Oversold. It's not Turing's fault, though. AI Blog (2021, was #1 on Hacker News) [10] J.S. Intelligente Roboter werden vom Leben fasziniert sein. (Intelligent robots will be fascinated by life.) F.A.Z., 2015 [11] J.S. at Falling Walls: The Past, Present and Future of Artificial Intelligence. Scientific American, Observations, 2017. [12] J.S. KI ist eine Riesenchance für Deutschland. (AI is a huge chance for Germany.) F.A.Z., 2018 [13] H. Jones. J.S. Says His Life's Work Won't Lead To Dystopia. Forbes Magazine, 2023. [14] Interview with J.S. Jazzyear, Shanghai, 2024. [15] J.S. TED talk at TED AI Vienna (2024): Why 2042 will be a big year for AI. See the attached video clip. [16] J.S. Baut den KI-gesteuerten Allzweckroboter! (Build the AI-controlled all-purpose robot!) F.A.Z., 2024 [17] J.S. 1995-2025: The Decline of Germany & Japan vs US & China. Can All-Purpose Robots Fuel a Comeback? AI Blog, Jan 2025, based on [16]. [18] M. Alhakami, D. R. Ashley, J. Dunham, Y. Dai, F. Faccio, E. Feron, J. Schmidhuber. Towards an Extremely Robust Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms. Preprint arxiv 2404.08093, 2024.
1:41

AGI? One day, but not yet. The only AI that works well right now is the one behind the screen [12-17]. But passing the Turing Test [9] behind a screen is easy compared to Real AI for real robots in the real world. No current AI-driven robot could be certified as a plumber [13-17]. Hence, the Turing Test isn't a good measure of intelligence (and neither is IQ). And AGI without mastery of the physical world is no AGI. That’s why I created the TUM CogBotLab for learning robots in 2004 [5], co-founded a company for AI in the physical world in 2014 [6], and had teams at TUM, IDSIA, and now KAUST work towards baby robots [4,10-11,18]. Such soft robots don't just slavishly imitate humans and they don't work by just downloading the web like LLMs/VLMs. No. Instead, they exploit the principles of Artificial Curiosity to improve their neural World Models (two terms I used back in 1990 [1-4]). These robots work with lots of sensors, but only weak actuators, such that they cannot easily harm themselves [18] when they collect useful data by devising and running their own self-invented experiments. Remarkably, since the 1970s, many have made fun of my old goal to build a self-improving AGI smarter than myself and then retire. Recently, however, many have finally started to take this seriously, and now some of them are suddenly TOO optimistic. These people are often blissfully unaware of the remaining challenges we have to solve to achieve Real AI. My 2024 TED talk [15] summarises some of that. REFERENCES (easy to find on the web): [1] J. Schmidhuber. Making the world differentiable: On using fully recurrent self-supervised neural networks (NNs) for dynamic reinforcement learning and planning in non-stationary environments. TR FKI-126-90, TUM, Feb 1990, revised Nov 1990. This paper also introduced artificial curiosity and intrinsic motivation through generative adversarial networks where a generator NN is fighting a predictor NN in a minimax game. [2] J. S. A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer and S. W. Wilson, editors, Proc. of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pages 222-227. MIT Press/Bradford Books, 1991. Based on [1]. [3] J.S. AI Blog (2020). 1990: Planning & Reinforcement Learning with Recurrent World Models and Artificial Curiosity. Summarising aspects of [1][2] and lots of later papers including [7][8]. [4] J.S. AI Blog (2021): Artificial Curiosity & Creativity Since 1990. Summarising aspects of [1][2] and lots of later papers including [7][8]. [5] J.S. TU Munich CogBotLab for learning robots (2004-2009) [6] NNAISENSE, founded in 2014, for AI in the physical world [7] J.S. (2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning (RL) Controllers and Recurrent Neural World Models. arXiv 1210.0118. Sec. 5.3 describes an RL prompt engineer which learns to query its model for abstract reasoning and planning and decision making. Today this is called "chain of thought." [8] J.S. (2018). One Big Net For Everything. arXiv 1802.08864. See also patent US11853886B2 and my DeepSeek tweet: DeepSeek uses elements of the 2015 reinforcement learning prompt engineer [7] and its 2018 refinement [8] which collapses the RL machine and world model of [7] into a single net. This uses my neural net distillation procedure of 1991: a distilled chain of thought system. [9] J.S. Turing Oversold. It's not Turing's fault, though. AI Blog (2021, was #1 on Hacker News) [10] J.S. Intelligente Roboter werden vom Leben fasziniert sein. (Intelligent robots will be fascinated by life.) F.A.Z., 2015 [11] J.S. at Falling Walls: The Past, Present and Future of Artificial Intelligence. Scientific American, Observations, 2017. [12] J.S. KI ist eine Riesenchance für Deutschland. (AI is a huge chance for Germany.) F.A.Z., 2018 [13] H. Jones. J.S. Says His Life's Work Won't Lead To Dystopia. Forbes Magazine, 2023. [14] Interview with J.S. Jazzyear, Shanghai, 2024. [15] J.S. TED talk at TED AI Vienna (2024): Why 2042 will be a big year for AI. See the attached video clip. [16] J.S. Baut den KI-gesteuerten Allzweckroboter! (Build the AI-controlled all-purpose robot!) F.A.Z., 2024 [17] J.S. 1995-2025: The Decline of Germany & Japan vs US & China. Can All-Purpose Robots Fuel a Comeback? AI Blog, Jan 2025, based on [16]. [18] M. Alhakami, D. R. Ashley, J. Dunham, Y. Dai, F. Faccio, E. Feron, J. Schmidhuber. Towards an Extremely Robust Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms. Preprint arxiv 2404.08093, 2024.

Jürgen Schmidhuber

72,331 views • 1 year ago

Self-Evolving AI : New MIT AI Rewrites its Own Code and it’s Changing Everything | Julian Horsey, Geeky Gadgets TL;DR Key Takeaways : - MIT’s SEAL framework introduces “self-adapting language models” that autonomously enhance their capabilities by generating synthetic training data, self-editing, and updating internal parameters. - SEAL’s self-adaptation process mirrors human learning, allowing continuous improvement and dynamic adaptation to new tasks without relying on external datasets. - Reinforcement learning serves as a feedback mechanism in SEAL, rewarding effective self-edits and making sure sustained progress and goal alignment. SEAL overcomes AI’s reliance on pre-existing datasets by generating its own training material, excelling in long-term task retention and complex problem-solving scenarios. - Potential applications of SEAL include autonomous robotics, personalized education, and advanced problem-solving in fields like healthcare, logistics, and scientific research. --- What if artificial intelligence could not only learn but also rewrite its own code to become smarter over time? This is no longer a futuristic fantasy—MIT’s new “self-adapting language models” (SEAL) framework has made it a reality. Unlike traditional AI systems that rely on external datasets and human intervention to improve, SEAL takes a bold leap forward by autonomously generating its own training data and refining its internal processes. In essence, this AI doesn’t just evolve—it rewires itself, mirroring the way humans adapt through trial, error, and self-reflection. The implications are staggering: a system that can independently enhance its capabilities could redefine the boundaries of what AI can achieve, from solving complex problems to adapting in real time to unforeseen challenges. In this exploration by Wes Roth of MIT’s innovative SEAL framework, you’ll uncover how this self-improving AI works and why it’s a fantastic option for the field of artificial intelligence. From its ability to overcome the “data wall” that limits many current systems to its use of reinforcement learning as a feedback mechanism, SEAL introduces a level of autonomy and adaptability that was previously unimaginable. Imagine AI systems that can retain knowledge over time, dynamically adjust to new tasks, and operate with minimal human oversight. Whether you’re intrigued by its potential for autonomous robotics, personalized education, or advanced problem-solving, SEAL’s ability to rewrite its own rules promises to reshape the future of technology. Could this be the first step toward truly independent, self-evolving AI? What Sets SEAL Apart? The SEAL framework introduces a novel concept of self-adaptation, distinguishing it from traditional AI models. Unlike conventional systems that depend on external datasets for updates, SEAL enables AI to generate synthetic training data independently. This self-generated data is then used to iteratively refine the model, making sure continuous improvement. By persistently updating its internal parameters, SEAL enables AI systems to dynamically adapt to new tasks and inputs. To better illustrate this, consider how humans learn. When faced with a new concept, you might take notes, revisit them, and refine your understanding as you gather more information. SEAL mirrors this process by continuously refining its internal knowledge and performance through iterative self-improvement. This capability allows SEAL to evolve in real time, making it uniquely suited for tasks requiring adaptability and long-term learning. The Role of Reinforcement Learning in SEAL Reinforcement learning plays a critical role in the SEAL framework, acting as a feedback mechanism that evaluates the effectiveness of the model’s self-edits. It rewards changes that enhance performance, creating a cycle of continuous improvement. Over time, this feedback loop optimizes the system’s ability to generate and apply edits, making sure sustained progress. This process is analogous to how humans learn through trial and error. By rewarding effective changes, SEAL aligns its self-generated data and edits with desired outcomes. The integration of reinforcement learning not only enhances the system’s adaptability but also ensures it remains focused on achieving specific goals. This structured feedback mechanism is a cornerstone of SEAL’s ability to refine itself autonomously and efficiently. Real-World Applications and Testing SEAL has demonstrated remarkable performance across various applications, particularly in tasks requiring the integration of factual knowledge and advanced question-answering capabilities. For instance, when tested on benchmarks like the ARC AGI, SEAL outperformed other models by effectively generating and using synthetic data. This ability to create its own training material addresses a significant limitation of current AI systems: their reliance on pre-existing datasets. SEAL’s capacity for long-term task retention and dynamic adaptation further enhances its utility. It excels in scenarios that demand sustained focus and coherence, such as answering complex questions or adapting to evolving objectives. By using its iterative learning process, SEAL is equipped to handle these challenges with exceptional efficiency, making it a valuable tool for a wide range of real-world applications. Overcoming AI’s Data Limitations One of SEAL’s most promising features is its ability to overcome the “data wall” that constrains many AI systems today. By generating synthetic data, SEAL ensures a continuous supply of training material, allowing sustained development without relying on external datasets. This capability is particularly valuable for autonomous AI systems that must operate independently over extended periods. Additionally, SEAL addresses a critical weakness in many current AI models: their struggle with coherence and task retention over long durations. By emulating human learning processes, SEAL enables AI systems to manage complex, long-term tasks with minimal human intervention. This ability to retain and apply knowledge over time positions SEAL as a fantastic tool for advancing AI capabilities. Potential Applications and Future Impact The introduction of SEAL marks a significant milestone in AI research, opening new possibilities for self-improving systems. Its ability to dynamically adapt, retain knowledge, and generate its own training data has far-reaching implications for the future of AI development. Potential applications include: - Autonomous robotics: Systems that can adapt to changing environments and perform tasks with minimal human oversight. - Personalized education: AI-driven platforms that tailor learning experiences to individual needs and preferences. - Advanced problem-solving: Applications in fields such as healthcare, logistics, and scientific research, where adaptability and precision are critical. Read more:
21:31

Self-Evolving AI : New MIT AI Rewrites its Own Code and it’s Changing Everything | Julian Horsey, Geeky Gadgets TL;DR Key Takeaways : - MIT’s SEAL framework introduces “self-adapting language models” that autonomously enhance their capabilities by generating synthetic training data, self-editing, and updating internal parameters. - SEAL’s self-adaptation process mirrors human learning, allowing continuous improvement and dynamic adaptation to new tasks without relying on external datasets. - Reinforcement learning serves as a feedback mechanism in SEAL, rewarding effective self-edits and making sure sustained progress and goal alignment. SEAL overcomes AI’s reliance on pre-existing datasets by generating its own training material, excelling in long-term task retention and complex problem-solving scenarios. - Potential applications of SEAL include autonomous robotics, personalized education, and advanced problem-solving in fields like healthcare, logistics, and scientific research. --- What if artificial intelligence could not only learn but also rewrite its own code to become smarter over time? This is no longer a futuristic fantasy—MIT’s new “self-adapting language models” (SEAL) framework has made it a reality. Unlike traditional AI systems that rely on external datasets and human intervention to improve, SEAL takes a bold leap forward by autonomously generating its own training data and refining its internal processes. In essence, this AI doesn’t just evolve—it rewires itself, mirroring the way humans adapt through trial, error, and self-reflection. The implications are staggering: a system that can independently enhance its capabilities could redefine the boundaries of what AI can achieve, from solving complex problems to adapting in real time to unforeseen challenges. In this exploration by Wes Roth of MIT’s innovative SEAL framework, you’ll uncover how this self-improving AI works and why it’s a fantastic option for the field of artificial intelligence. From its ability to overcome the “data wall” that limits many current systems to its use of reinforcement learning as a feedback mechanism, SEAL introduces a level of autonomy and adaptability that was previously unimaginable. Imagine AI systems that can retain knowledge over time, dynamically adjust to new tasks, and operate with minimal human oversight. Whether you’re intrigued by its potential for autonomous robotics, personalized education, or advanced problem-solving, SEAL’s ability to rewrite its own rules promises to reshape the future of technology. Could this be the first step toward truly independent, self-evolving AI? What Sets SEAL Apart? The SEAL framework introduces a novel concept of self-adaptation, distinguishing it from traditional AI models. Unlike conventional systems that depend on external datasets for updates, SEAL enables AI to generate synthetic training data independently. This self-generated data is then used to iteratively refine the model, making sure continuous improvement. By persistently updating its internal parameters, SEAL enables AI systems to dynamically adapt to new tasks and inputs. To better illustrate this, consider how humans learn. When faced with a new concept, you might take notes, revisit them, and refine your understanding as you gather more information. SEAL mirrors this process by continuously refining its internal knowledge and performance through iterative self-improvement. This capability allows SEAL to evolve in real time, making it uniquely suited for tasks requiring adaptability and long-term learning. The Role of Reinforcement Learning in SEAL Reinforcement learning plays a critical role in the SEAL framework, acting as a feedback mechanism that evaluates the effectiveness of the model’s self-edits. It rewards changes that enhance performance, creating a cycle of continuous improvement. Over time, this feedback loop optimizes the system’s ability to generate and apply edits, making sure sustained progress. This process is analogous to how humans learn through trial and error. By rewarding effective changes, SEAL aligns its self-generated data and edits with desired outcomes. The integration of reinforcement learning not only enhances the system’s adaptability but also ensures it remains focused on achieving specific goals. This structured feedback mechanism is a cornerstone of SEAL’s ability to refine itself autonomously and efficiently. Real-World Applications and Testing SEAL has demonstrated remarkable performance across various applications, particularly in tasks requiring the integration of factual knowledge and advanced question-answering capabilities. For instance, when tested on benchmarks like the ARC AGI, SEAL outperformed other models by effectively generating and using synthetic data. This ability to create its own training material addresses a significant limitation of current AI systems: their reliance on pre-existing datasets. SEAL’s capacity for long-term task retention and dynamic adaptation further enhances its utility. It excels in scenarios that demand sustained focus and coherence, such as answering complex questions or adapting to evolving objectives. By using its iterative learning process, SEAL is equipped to handle these challenges with exceptional efficiency, making it a valuable tool for a wide range of real-world applications. Overcoming AI’s Data Limitations One of SEAL’s most promising features is its ability to overcome the “data wall” that constrains many AI systems today. By generating synthetic data, SEAL ensures a continuous supply of training material, allowing sustained development without relying on external datasets. This capability is particularly valuable for autonomous AI systems that must operate independently over extended periods. Additionally, SEAL addresses a critical weakness in many current AI models: their struggle with coherence and task retention over long durations. By emulating human learning processes, SEAL enables AI systems to manage complex, long-term tasks with minimal human intervention. This ability to retain and apply knowledge over time positions SEAL as a fantastic tool for advancing AI capabilities. Potential Applications and Future Impact The introduction of SEAL marks a significant milestone in AI research, opening new possibilities for self-improving systems. Its ability to dynamically adapt, retain knowledge, and generate its own training data has far-reaching implications for the future of AI development. Potential applications include: - Autonomous robotics: Systems that can adapt to changing environments and perform tasks with minimal human oversight. - Personalized education: AI-driven platforms that tailor learning experiences to individual needs and preferences. - Advanced problem-solving: Applications in fields such as healthcare, logistics, and scientific research, where adaptability and precision are critical. Read more:

Owen Gregorian

70,672 views • 1 year ago

The best undergraduate and graduate research awards at Caltech commencement this year were presented to research on Neural Operators by my students @ZongyiLiCaltech and Miguel Liu-Schiaffini. This recognition is a testament to their passion and dedication, and the impact their work has already had: from building a foundation for AI+Science through multi-scale learning, creating the first high-resolution AI-based weather model, designing a novel medical catheter that reduces bacterial contamination, to name just a few. The future of Physical AI will be built on the foundation of Neural Operators since the real world exists in multiple scales and resolutions, while standard deep learning only supports fixed resolution. This is essential for us to unlock the full potential of AI for scientific discovery. #Caltech2025 #ai #science #NeuralOperators
4:09

The best undergraduate and graduate research awards at Caltech commencement this year were presented to research on Neural Operators by my students @ZongyiLiCaltech and Miguel Liu-Schiaffini. This recognition is a testament to their passion and dedication, and the impact their work has already had: from building a foundation for AI+Science through multi-scale learning, creating the first high-resolution AI-based weather model, designing a novel medical catheter that reduces bacterial contamination, to name just a few. The future of Physical AI will be built on the foundation of Neural Operators since the real world exists in multiple scales and resolutions, while standard deep learning only supports fixed resolution. This is essential for us to unlock the full potential of AI for scientific discovery. #Caltech2025 #ai #science #NeuralOperators

Prof. Anima Anandkumar

29,765 views • 1 year ago

This morning at NeurIPS, Rich Sutton reminded us that we need continual learning to reach AGI. This afternoon, Ali Behrouz presented a Google poster paper, Nested Learning, which provides new ideas on the path to continual learning. I recorded the 40 minute talk as it might be useful for some researchers in the audience. For the rest of us, I subscribe to Andrej Karpathy's suspicion that it will take a 5-10 papers like this to move us to AGI from where we are now, just like it took about 10 papers to move from 2012's AlexNet to ChatGPT. At the very end, I ask Ali how far along to continual learning this represents. Full paper link below, as well as a YouTube link. ps. sorry about the first 2 minutes of bad audio since there were 2 idiots standing beside me have a conversation right in front of this presenter in a rather packed poster presentation. Honestly, tamp down your egos guys and show come common courtesy!
40:27

This morning at NeurIPS, Rich Sutton reminded us that we need continual learning to reach AGI. This afternoon, Ali Behrouz presented a Google poster paper, Nested Learning, which provides new ideas on the path to continual learning. I recorded the 40 minute talk as it might be useful for some researchers in the audience. For the rest of us, I subscribe to Andrej Karpathy's suspicion that it will take a 5-10 papers like this to move us to AGI from where we are now, just like it took about 10 papers to move from 2012's AlexNet to ChatGPT. At the very end, I ask Ali how far along to continual learning this represents. Full paper link below, as well as a YouTube link. ps. sorry about the first 2 minutes of bad audio since there were 2 idiots standing beside me have a conversation right in front of this presenter in a rather packed poster presentation. Honestly, tamp down your egos guys and show come common courtesy!

Phil Trubey

235,587 views • 8 months ago

HRT, one of the largest quantitative trading firms in the world, just revealed the scale of their AI operation. "two of the biggest ingredients you need to build deep learning models are compute and data. HRT has both in incredible orders of magnitude." "we have vast quantities of data, on the order of petabytes, just in terms of raw tick data we've written down since the beginning of electronic trading." "we have a very high compute-per-researcher ratio." then the line that puts it in perspective: "there probably aren't too many places on Earth you can do cutting-edge deep learning research at the level we do it, just because of how few places have access to as much compute." "it's basically the hyperscalers of the world and then we're in the next tranche." a trading firm putting itself in the same compute tier as Google, Microsoft, and Meta. not for chatbots, for trading.
2:08

HRT, one of the largest quantitative trading firms in the world, just revealed the scale of their AI operation. "two of the biggest ingredients you need to build deep learning models are compute and data. HRT has both in incredible orders of magnitude." "we have vast quantities of data, on the order of petabytes, just in terms of raw tick data we've written down since the beginning of electronic trading." "we have a very high compute-per-researcher ratio." then the line that puts it in perspective: "there probably aren't too many places on Earth you can do cutting-edge deep learning research at the level we do it, just because of how few places have access to as much compute." "it's basically the hyperscalers of the world and then we're in the next tranche." a trading firm putting itself in the same compute tier as Google, Microsoft, and Meta. not for chatbots, for trading.

Goshawk Trades

14,548 views • 2 months ago

Sam Altman on what you need to do to survive in the age of artificial intelligence. "You are about to enter the greatest golden age of human possibility..." To thrive in that world, the skills that matter most are: - Deep familiarity with the tools - Staying abreast of changes - Developing a great intuition for AI tools, where things are going, and how to make use of it - Resilience and the ability to learn things fast and evolve yourself with technology I know most of this stuff is a pretty big no-brainer for anyone paying attention, but here's the takeaway: AI upskilling and keeping up with AI are possibly the most important skills in the world right now. And the most fascinating part is that new AI developments and tools come out so fast that everyone is constantly learning. I'd be willing to bet that if you're a complete beginner but spend 3 months of dedicated learning, you'd be fully caught up to power users. In other words: Spend 3 months learning → work 2-4x faster and future-proof yourself. There's so much opportunity for anyone who just starts now.
0:38

Sam Altman on what you need to do to survive in the age of artificial intelligence. "You are about to enter the greatest golden age of human possibility..." To thrive in that world, the skills that matter most are: - Deep familiarity with the tools - Staying abreast of changes - Developing a great intuition for AI tools, where things are going, and how to make use of it - Resilience and the ability to learn things fast and evolve yourself with technology I know most of this stuff is a pretty big no-brainer for anyone paying attention, but here's the takeaway: AI upskilling and keeping up with AI are possibly the most important skills in the world right now. And the most fascinating part is that new AI developments and tools come out so fast that everyone is constantly learning. I'd be willing to bet that if you're a complete beginner but spend 3 months of dedicated learning, you'd be fully caught up to power users. In other words: Spend 3 months learning → work 2-4x faster and future-proof yourself. There's so much opportunity for anyone who just starts now.

Rowan Cheung

854,262 views • 2 years ago

School outside of school. AI-first style. When we all don't have jobs (I don't believe that, by the way, but let's run with that idea) what will we do? You'll take your kids to school in an autonomous car. Put on your glasses that give you big digital displays. And learn something in the parking lot. Those with electric vehicles are already able to do a lot of this (can't sit in a gas car on a hot day without running the engine, and that's not good for it). On a hot day sometimes I just drive somewhere interesting and sit in my car working in my Apple Vision Pro. Here I'm sitting by my son's high school. My brain doesn't take to languages easily. Probably because my mom was German and yelled at me in German. So I resisted learning. But she has been dead for a decade, so am trying to retrain my brain to be more interesting than be controlled by childhood trauma. Using A new app that uses AI to talk to me in Spanish, along with a variety of languages. My wife moved to the USA from Tehran, Iran, when she was 16 and attended high school here in Silicon Valley. Didn't know a bit of English when she moved here. She learned by watching TV. TalkMe does something different than other apps I've used and other teaching methodologies. It just has a conversation with me. It speaks to me in Spanish and underneath it shows me the English translation, so I understand what it's trying to say to me. And that alone lets me learn about the diction, about the vocabulary, about the pronunciation. Then it asks me to say a Spanish response. I just read off the screen and say it, but what's going on in my mind is I'm actually learning the words and learning the language and learning in a way my wife just couldn't learn by watching TV. It gets to the point that AI is changing how we learn. By having conversations with us we can learn any topic and learn it in the style that your brain can process best. I've been using it for hours this week while my car has driven me back and forth to San Francisco a few times (busy week) and it hit me, AI isn't just for driving my car. It's for teaching me Spanish, and, well, anything. And when we don't have a job? Maybe we'll have to learn something new to do with our lives. I'm so excited by what AI can do for our education system. And if the school doesn't use it, well, sit out in the parking lot and learn faster than inside! Here you go hearing how bad my Spanish is. But I am already getting better. See you next time from the AI-first chronicles.
3:57

School outside of school. AI-first style. When we all don't have jobs (I don't believe that, by the way, but let's run with that idea) what will we do? You'll take your kids to school in an autonomous car. Put on your glasses that give you big digital displays. And learn something in the parking lot. Those with electric vehicles are already able to do a lot of this (can't sit in a gas car on a hot day without running the engine, and that's not good for it). On a hot day sometimes I just drive somewhere interesting and sit in my car working in my Apple Vision Pro. Here I'm sitting by my son's high school. My brain doesn't take to languages easily. Probably because my mom was German and yelled at me in German. So I resisted learning. But she has been dead for a decade, so am trying to retrain my brain to be more interesting than be controlled by childhood trauma. Using A new app that uses AI to talk to me in Spanish, along with a variety of languages. My wife moved to the USA from Tehran, Iran, when she was 16 and attended high school here in Silicon Valley. Didn't know a bit of English when she moved here. She learned by watching TV. TalkMe does something different than other apps I've used and other teaching methodologies. It just has a conversation with me. It speaks to me in Spanish and underneath it shows me the English translation, so I understand what it's trying to say to me. And that alone lets me learn about the diction, about the vocabulary, about the pronunciation. Then it asks me to say a Spanish response. I just read off the screen and say it, but what's going on in my mind is I'm actually learning the words and learning the language and learning in a way my wife just couldn't learn by watching TV. It gets to the point that AI is changing how we learn. By having conversations with us we can learn any topic and learn it in the style that your brain can process best. I've been using it for hours this week while my car has driven me back and forth to San Francisco a few times (busy week) and it hit me, AI isn't just for driving my car. It's for teaching me Spanish, and, well, anything. And when we don't have a job? Maybe we'll have to learn something new to do with our lives. I'm so excited by what AI can do for our education system. And if the school doesn't use it, well, sit out in the parking lot and learn faster than inside! Here you go hearing how bad my Spanish is. But I am already getting better. See you next time from the AI-first chronicles.

Robert Scoble

131,040 views • 11 months ago