Drop 3/15: Introducing the Speech Recognition Benchmark on Voice Arena. One benchmark for every use case of speech recognition, in every language. Tested on real conversations. Dataset 1 is live: the Monsoon Conversational ASR Test Set, two-person conversations in 5 languages. Leaderboard and public test sets at Voice Arena.

The leaderboards: Top 12 models on each of the five language boards. 41 models are scored on US English, 39 on Indian English, 37 on Brazilian Portuguese, 27 on Romanian and 24 on Hindi. Full boards at

Tested on real-world audio, not studio recordings. Every clip is a real phone call: everyday phones, real rooms, real background sound. Audio quality varies by language. Hindi is the hardest, with 9% of private clips below 3.0 MOS. The public sets match the private sets in audio quality, within 0.03 MOS in every language.

Languages for billions of people. The first five languages : US English, Brazilian Portuguese, Hindi, Indian English and Romanian, have over 2 billion speakers between them. Each test set was recorded in-country. The goal is 100+ languages. Public test sets on Hugging Face: Leaderboard live at

@voicearena_ai Great work, team.

@voicearena_ai The future is here.

@voicearena_ai Are the speakers in the public and private sets different people?

@voicearena_ai Big release. Congrats to the team.

@voicearena_ai The streaming results are the most interesting part for me.

@voicearena_ai Open models being this close on English is a big shift.

@voicearena_ai Will you add more languages to the public sets over time?

@voicearena_ai Will rerun this whenever our provider ships a new model.

@voicearena_ai This answers a question our team argues about every quarter.

@voicearena_ai Great to see Brazilian Portuguese and Romanian covered.

@voicearena_ai Real-world audio benchmarks should be the norm, not the exception.

@voicearena_ai Our team will use the gender breakdown for bias checks.

@voicearena_ai Which language is coming next?

@voicearena_ai Needed this.

@voicearena_ai Nice to see public test sets released alongside.

@voicearena_ai The best model on a language matters less than the best model for your callers. The filters get at that.

@voicearena_ai Nice that the charts say where the axis starts. Small thing, but honest.

@voicearena_ai The streaming results alone are worth sharing internally.

@voicearena_ai Will help a lot for multilingual rollouts.

@voicearena_ai Will be interesting to see how vendors respond to this.

@voicearena_ai There's a lot of signal in the noise-level breakdown.

@voicearena_ai Matched public and private sets is a clever way to keep things reproducible and fair.

@voicearena_ai Refreshing to see "scores compare within a language only" stated upfront.

@voicearena_ai This is the shortlist I'll start from next time.

@voicearena_ai Sharing this with our speech team.

@voicearena_ai Great resource for anyone starting out with voice agents.

@voicearena_ai Is there a way to submit our own model for evaluation?

@voicearena_ai Love this.

@voicearena_ai Accuracy on real calls is what our customers notice. Glad it's measured.

@voicearena_ai The spread on the Romanian board shows how uneven language support still is.

@voicearena_ai The noisy-call slice is what our support calls actually sound like.

@voicearena_ai Our users are mostly on phones, so this is the relevant benchmark for us.

@voicearena_ai Really well done.

@voicearena_ai Instant bookmark.