MIPROv2, our new state-of-the-art optimizer for LM programs, is live in DSPy Stanford NLP Group! It's even faster, cheaper, and more accurate than MIPRO. MIPROv2 proposes instructions, bootstraps demonstrations, and optimizes combinations. Let’s dive into a visual 🧵of how it works!

First, MIPROv2 tries to understand your task. It reads your DSPy code, analyzes your dataset, and runs your program a few times to produce example traces.  These will inform the proposal LM to write better and more grounded instructions!

Next, MIPROv2 generates demonstrations of your program. It runs your program several times and keeps traces whose output is scored highly.  These traces form the basis for optimizing few-shot demonstrations.  Besides what your metric requires, no labels necessary!

Given the instructions & demonstrations it proposed, the MIPROs build a Bayesian surrogate model to sample combinations and assign a belief over their utility. To make MIPROv2 faster & cheaper than v1, the updates happen on small *mini-batches* of your data.

MIPROv2 is already live in DSPy 2.4.10+! Check out this notebook, where we show how to optimize a program for ScoNe, an NLI benchmark with just over 1000 LM calls — an order of magnitude fewer than MIPROv1!

If you want more details, read our paper release thread or join the community at Joint work with an amazing team: @kristahopsalong @JoshPurtell @DavidKarlBroman @matei_zaharia @ChrisGPotts @lateinteraction

Amazing release, @michaelryan207. These are some of the best visuals I saw in a very long time!

@stanfordnlp Can't wait to try this

@stanfordnlp What tooling did you use to make these visuals? The edit and story is quite high quality

@stanfordnlp Thanks Nirant! The visuals were all produced in Keynote

@stanfordnlp Awesome work as always! Looking forward to trying this out.