Skip navigation
MIPROv2, our new state-of-the-art optimizer for LM programs, is live in DSPy Stanford NLP Group!
0:00 / 1:34

MIPROv2, our new state-of-the-art optimizer for LM programs, is live in DSPy Stanford NLP Group!

Michael Ryan

@michaelryan2072.4K followers

Visit on X
496 likesDownloadOriginal post

156,809 views

MIPROv2, our new state-of-the-art optimizer for LM programs, is live in DSPy Stanford NLP Group! It's even faster, cheaper, and more accurate than MIPRO. MIPROv2 proposes instructions, bootstraps demonstrations, and optimizes combinations. Let’s dive into a visual 🧵of how it works!

10 Comments

Comments from the original post on X (Twitter)

  • Michael Ryan's profile picture

    @michaelryan207

    First, MIPROv2 tries to understand your task. It reads your DSPy code, analyzes your dataset, and runs your program a few times to produce example traces. These will inform the proposal LM to write better and more grounded instructions!

    34 likes

  • Michael Ryan's profile picture

    @michaelryan207

    Next, MIPROv2 generates demonstrations of your program. It runs your program several times and keeps traces whose output is scored highly. These traces form the basis for optimizing few-shot demonstrations. Besides what your metric requires, no labels necessary!

    26 likes

  • Michael Ryan's profile picture

    @michaelryan207

    Given the instructions & demonstrations it proposed, the MIPROs build a Bayesian surrogate model to sample combinations and assign a belief over their utility. To make MIPROv2 faster & cheaper than v1, the updates happen on small *mini-batches* of your data.

    23 likes

  • Michael Ryan's profile picture

    @michaelryan207

    MIPROv2 is already live in DSPy 2.4.10+! Check out this notebook, where we show how to optimize a program for ScoNe, an NLI benchmark with just over 1000 LM calls — an order of magnitude fewer than MIPROv1!

    41 likes

  • Michael Ryan's profile picture

    @michaelryan207

    If you want more details, read our paper release thread or join the community at Joint work with an amazing team: @kristahopsalong @JoshPurtell @DavidKarlBroman @matei_zaharia @ChrisGPotts @lateinteraction

    34 likes

  • Omar Khattab's profile picture

    @lateinteraction

    Amazing release, @michaelryan207. These are some of the best visuals I saw in a very long time!

    13 likes

  • Mike Taylor's profile picture

    @hammer_mt

    @stanfordnlp Can't wait to try this

    2 likes

  • Nirant's profile picture

    @NirantK

    @stanfordnlp What tooling did you use to make these visuals? The edit and story is quite high quality

    2 likes

  • Michael Ryan's profile picture

    @michaelryan207

    @stanfordnlp Thanks Nirant! The visuals were all produced in Keynote

    3 likes

  • Karthik Kalyanaraman's profile picture

    @karthikkalyan90

    @stanfordnlp Awesome work as always! Looking forward to trying this out.

    1 like