we made AI structured outputs 1 MILLION times faster

is this a smaller model finetuned on structured data or is there something unique u guys are also pushing?

it's actually a larger model, somewhat similar to the way sparse mixture of depths/experts works but with millions of individual shards that we project into smaller submodels at runtime

i believe the scientific term for this is “absolute banger”

The benchmark are against gpt-4-turbo and 3-turbo. A better test could be with gpt-4o-2024-08-06 which implements structured outputs:

OpenAI's structured outputs mode only supports ~5% of our benchmarks, since we use a lot of features they don't support (like optional fields, regex strings, format strings like emails and ISO datetime/date, numbers with min/max, etc.) and because they error on large schemas.

@rauchg this seems perfect for generative UI and AI SDK ?

@rauchg indeed it is

based

Fucking 🔥

amazing