MIPROv2, our new state-of-the-art optimizer for LM programs, is live in DSPy Stanford NLP Group!
@michaelryan2072.4K followers
156,809 views
10 Comments
Comments from the original post on X (Twitter)

@michaelryan207
First, MIPROv2 tries to understand your task. It reads your DSPy code, analyzes your dataset, and runs your program a few times to produce example traces. These will inform the proposal LM to write better and more grounded instructions!
34 likes

@michaelryan207
Next, MIPROv2 generates demonstrations of your program. It runs your program several times and keeps traces whose output is scored highly. These traces form the basis for optimizing few-shot demonstrations. Besides what your metric requires, no labels necessary!
26 likes

@michaelryan207
Given the instructions & demonstrations it proposed, the MIPROs build a Bayesian surrogate model to sample combinations and assign a belief over their utility. To make MIPROv2 faster & cheaper than v1, the updates happen on small *mini-batches* of your data.
23 likes

@michaelryan207
MIPROv2 is already live in DSPy 2.4.10+! Check out this notebook, where we show how to optimize a program for ScoNe, an NLI benchmark with just over 1000 LM calls — an order of magnitude fewer than MIPROv1!
41 likes

@michaelryan207
If you want more details, read our paper release thread or join the community at Joint work with an amazing team: @kristahopsalong @JoshPurtell @DavidKarlBroman @matei_zaharia @ChrisGPotts @lateinteraction
34 likes

@lateinteraction
Amazing release, @michaelryan207. These are some of the best visuals I saw in a very long time!
13 likes

@hammer_mt
@stanfordnlp Can't wait to try this
2 likes

@NirantK
@stanfordnlp What tooling did you use to make these visuals? The edit and story is quite high quality
2 likes

@michaelryan207
@stanfordnlp Thanks Nirant! The visuals were all produced in Keynote
3 likes

@karthikkalyan90
@stanfordnlp Awesome work as always! Looking forward to trying this out.
1 like