Introducing Predicted Outputs—dramatically decrease latency for gpt-4o and gpt-4o-mini by providing a reference string. Speed up: - Updating a blog post in a doc - Iterating on prior responses - Rewriting code in an existing file, like Exponent here:

See @FactoryAI's results:

@exponent_run Will this fix the GPT-4o repeating the same code back with no changes bug!?!? If you predict the previous code back in and specifically omit the commentary in the prediction, then I think it would have no choice but to edit the code!? Cuz it can’t edit the commentary??

@exponent_run curious to see how this will work with @cursor_ai's composer mode

@exponent_run @cursor_ai for the love of the almightly, we need this lol.

@exponent_run incredible work 🍓

@exponent_run This will be very useful on my for some of the internal steps, each with 60k+ tokens/call x dozens of calls/generated paper or book. Significant.

@exponent_run fastest way to make web apps with openai api:

@exponent_run Anthropic has left the chat

@exponent_run We can go faster.

@exponent_run Am I understanding this correctly that you are charged for the whole prediction text you give? So this improves latency but will still be just as costly as having it generate the entire output text?