Loading video...

Video Failed to Load

Go Home

Introducing Predicted Outputs—dramatically decrease latency for gpt-4o and gpt-4o-mini by providing a reference string. Speed up: - Updating a blog post in a doc - Iterating on prior responses - Rewriting code in an existing file, like Exponent here:

581,951 views • 2 years ago •via X (Twitter)

10 Comments

OpenAI Developers's profile picture
OpenAI Developers2 years ago

See @FactoryAI's results:

Nick Dobos's profile picture
Nick Dobos2 years ago

@exponent_run Will this fix the GPT-4o repeating the same code back with no changes bug!?!? If you predict the previous code back in and specifically omit the commentary in the prediction, then I think it would have no choice but to edit the code!? Cuz it can’t edit the commentary??

HudZah ⁂'s profile picture
HudZah ⁂2 years ago

@exponent_run curious to see how this will work with @cursor_ai's composer mode

The Canaanite's profile picture
The Canaanite2 years ago

@exponent_run @cursor_ai for the love of the almightly, we need this lol.

🍓🍓🍓's profile picture
🍓🍓🍓2 years ago

@exponent_run incredible work 🍓

Garrett of DeepwriterAI's profile picture
Garrett of DeepwriterAI2 years ago

@exponent_run This will be very useful on my for some of the internal steps, each with 60k+ tokens/call x dozens of calls/generated paper or book. Significant.

AK's profile picture
AK2 years ago

@exponent_run fastest way to make web apps with openai api:

Itay Bachman's profile picture
Itay Bachman2 years ago

@exponent_run Anthropic has left the chat

Pseudonym 🦅's profile picture
Pseudonym 🦅2 years ago

@exponent_run We can go faster.

Chase Brower's profile picture
Chase Brower2 years ago

@exponent_run Am I understanding this correctly that you are charged for the whole prediction text you give? So this improves latency but will still be just as costly as having it generate the entire output text?

Related Videos