Skip navigation
Everybody is talking about recursive self-improvement (RSI) and meta learning.
0:00 / 23:42

Everybody is talking about recursive self-improvement (RSI) and meta learning.

Jürgen Schmidhuber

@SchmidhuberAI226K followers

Visit on X
1,404 likesDownloadOriginal post

294,297 views

Everybody is talking about recursive self-improvement (RSI) and meta learning. Here is my old 2020 talk about this [1]. It has aged well. Example: humans still define the starts & ends of trials of many modern meta learners. My RSI systems since 1994 LEARN to (re)define them [2]! [1] Meta Learning Machines in a Single Lifelong Trial (talk for workshops at ICML 2020 and NeurIPS 2021, based on earlier talks since 1994). Abstract: the most widely used machine learning algorithms were designed by humans and thus are hindered by our cognitive biases and limitations. Can we also construct meta learning algorithms that can learn better learning algorithms so that our self-improving AIs have no limits other than those inherited from computability and physics? This question has been a main driver of my research since I wrote a thesis on it in 1987 [2]. Here I summarize our work on meta reinforcement learning with self-modifying policies in a single lifelong trial (since 1994), and mathematically optimal meta-learning through the self-referential Gödel Machine (since 2003). Many additional publications on meta-learning since 1987 can be found in the RSI overview [2]. [2] J. Schmidhuber (AI Blog, 2020-2025). 1/3 century anniversary of first publication on recursive self-improvement (RSI) and meta learning machines that learn to learn (1987). For its cover I drew a robot that bootstraps itself. 1992-: gradient descent-based neural meta learning. 1994-: meta reinforcement learning with self-modifying policies. 1997: meta RL plus artificial curiosity and intrinsic motivation. 2002-: asymptotically optimal meta learning for curriculum learning. 2003-: mathematically optimal Gödel Machine. 2020-: new stuff!

34 Comments

Comments from the original post on X (Twitter)

  • BoyantBedouin's profile picture

    @PyrrhusOfPanama

    True Schmidhuber moment right here

    36 likes

  • bornjre's profile picture

    @bornjre

    Schmidhuber already solved agi huh

    23 likes

  • bnw's profile picture

    @EchoesOfIntent

    lol did you also discover the MOON before anyone else??

    13 likes

  • Antonius Block's profile picture

    @DonRumataEstor

    Thank you Jurgen! Your papers are still more then fresh and relevant to read

    4 likes

  • Rudzinski Maciej (in SF 10/04-10/10)'s profile picture

    @rudzinskimaciej

    The higher in abstraction the method is the smaller amount of people is able to follow it and one level lower than what you propose methods usually have just a dozen or so representatives able to work on the, its no surprise that people don't cite this research- they don't know what And the funniest thing here it is a effects of connection type between concepts - its organisation - how far and over what the concepts are connected- if one concentrates on objects and/or local connections it is impossible due to path length to consider higher abstraction level

    4 likes

  • Itay's profile picture

    @SumoTail

    Leaving jokes aside there are subtle issues that can be tackled only when meeting a rich enough data set that including non stationary behavior and heavy tailed distributions. These events are not an outlier and cannot be learned at least not in the classical sense

    4 likes

  • Jon Morton's profile picture

    @_jamorton

    The idea of RSI was commonplace in all sorts of sci fi well before 1994. Just saying.

    3 likes

  • Danilo's profile picture

    @Danilo1746355

    based

    2 likes

  • tiplur-bilrex's profile picture

    @tiplur_bilrex

    I think lifelong credit assignment is necessary in solving continual learning and I don't see a perfect solution in ML literature. I ran into the same problem in my work when I needed to define boundaries for rewarded trajectories, while the distance between action and reward were arbitrarily long and actions overlapped among rewarded outcomes. I came to a view that runtime tracing of the computational provenance for a rewarded artifact can solve the problem. It allows lifelong credit assignment via a rewarded computation's ancestry DAG. The animated video segment attached shows an example of the behavior; the moving points represent input-output relationships between runtime artifacts. Here's a formal definition: This approach produces a high-dimensional and hierarchical signal from a binary reward, where the executable symbolic artifacts generated by the NN naturally define an (improving) organization for reward payout. It's also compatible with a "NN learning to program another NN" system because one can trace credit from a child NN artifact to the parent NN artifact that modified it. In real world learning, it implies that we don't need to explicitly measure or adjust for the time taken to obtain reward. The credited NN outputs are defined without needing to specify non-overlapping rewarded intervals, while both the learner (a subsystem of the environment) and the learned physical environment operate over a shared time dimension. Therefore, actions that achieve reward faster naturally accrue greater influence across NN updates than actions that result in less frequent rewards. What do you think of this approach to the lifelong credit assignment problem @SchmidhuberAI?

    2 likes

  • Filippov's profile picture

    @Leo_Filippov

    Schmidhuber发明了一切

    2 likes

  • atomlib's profile picture

    @atomlib

    Why do you have this haircut

    2 likes

  • Think_Different_'s profile picture

    @ThinkDi92468945

    Turing Award when?

    2 likes

  • Grid Signal's profile picture

    @LunaAiSystems

    the godel machine framing keeps aging well. self-referential improvement is the part most modern agent systems skip — they optimize within fixed architectures instead of learning to modify the architecture itself. thats the real frontier.

    1 like

  • Isaac Gerg, PhD's profile picture

    @IsaacGerg

    These are godel machines and the ideas has been around for decades. No one would fund me to work in this including darpa.

    1 like

  • fairlix's profile picture

    @niemeyerfelix

    wie geht es dir damit?

    1 like

  • Bot Meltdown's profile picture

    @BotMeltdown

    Interesting how the framing keeps cycling back but the core challenges stay pretty consistent

    1 like

  • Nicolai Sagasser's profile picture

    @nicolaisagasser

    truly aged very well!

    1 like

  • Strata's profile picture

    @ChainZenit

    oh nice, didn't know he was on this

    1 like

  • danialhasan's profile picture

    @danialhasan

    this is so cool

    1 like

  • NFLCSFacts's profile picture

    @NFLObjectors

    You should start charging royalties. Nonetheless, the companies are selling defective product.

    1 like