Shobhit Banga's banner

Shobhit Banga

@shobhitbanga • 3,391 subscribers

Scorekeeper for voice AI. Building independent, real-world evals to help labs build the best voice models. Co-founder @voicearena_ai | prev @JoshTalksAI

Shorts

Every language in Monsoon V1, in one spin of the globe. 50 languages. 23 countries. Spontaneous speech, not read prompts. If your ASR works in English and falls apart everywhere else, this is the data it was missing.

Every language in Monsoon V1, in one spin of the globe. 50 languages. 23 countries. Spontaneous speech, not read prompts. If your ASR works in English and falls apart everywhere else, this is the data it was missing.

173,286 Aufrufe

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Introducing Monsoon ASR ⚡️ Voice Arena Speech recognition does not have a model problem anymore. It has a data problem. The best ASR systems are approaching human-level performance in English. But move into the long tail of the world's languages, especially real, conversational speech and error rates can still be 5-10X higher. Today, we're releasing Monsoon ASR: a new generation of training data built specifically to close that gap. 50 languages. 100,000+ hours. Dense spontaneous speech. And one goal: Single-digit WER across the world's languages.🧵

Introducing Monsoon ASR ⚡️ Voice Arena Speech recognition does not have a model problem anymore. It has a data problem. The best ASR systems are approaching human-level performance in English. But move into the long tail of the world's languages, especially real, conversational speech and error rates can still be 5-10X higher. Today, we're releasing Monsoon ASR: a new generation of training data built specifically to close that gap. 50 languages. 100,000+ hours. Dense spontaneous speech. And one goal: Single-digit WER across the world's languages.🧵

648,443 Aufrufe • vor 28 Tagen

Introducing Diarization Bench, Voice Arena's benchmark for who spoke when. The metric we measure is diarization error rate, or DER. It measures how much of a conversation a model attaches to the wrong person. Every second it misses, invents, or gives to another speaker counts against it. Here's what we did. 139 conversations recorded for this board, in rooms and on calls, with people interrupting each other throughout. 12 systems scored under a single goal: testing models in the real-world conditions they are actually deployed in. NVIDIA's Nemotron 3 leads at 14.72%. 🧵

Introducing Diarization Bench, Voice Arena's benchmark for who spoke when. The metric we measure is diarization error rate, or DER. It measures how much of a conversation a model attaches to the wrong person. Every second it misses, invents, or gives to another speaker counts against it. Here's what we did. 139 conversations recorded for this board, in rooms and on calls, with people interrupting each other throughout. 12 systems scored under a single goal: testing models in the real-world conditions they are actually deployed in. NVIDIA's Nemotron 3 leads at 14.72%. 🧵

647,052 Aufrufe • vor 1 Monat

Introducing Jarvis Bench v0.5, Voice Arena's conversational agent benchmark. We've been obsessed with one question at VoiceArena: why do voice agent demos sound incredible, benchmarks say models are near-perfect, and yet you probably didn't have a single real conversation with a voice agent in the last 24 hours? Here's what's different about the Jarvis Bench. Real humans have live conversations with voice agents. A second group of humans blind-votes pairwise on two questions: which sounded more human, which got the job done. And one of the "agents" on the leaderboard is a human.

Introducing Jarvis Bench v0.5, Voice Arena's conversational agent benchmark. We've been obsessed with one question at VoiceArena: why do voice agent demos sound incredible, benchmarks say models are near-perfect, and yet you probably didn't have a single real conversation with a voice agent in the last 24 hours? Here's what's different about the Jarvis Bench. Real humans have live conversations with voice agents. A second group of humans blind-votes pairwise on two questions: which sounded more human, which got the job done. And one of the "agents" on the leaderboard is a human.

573,895 Aufrufe • vor 1 Monat

We’ve spent years collecting & annotating speech data. What became equally important was learning how to evaluate speech models in a way that reflects real listeners, real accents, & real usage. This video explains how we ran this blind evaluation & what @JoshTalksLive is building in speech evaluations.

We’ve spent years collecting & annotating speech data. What became equally important was learning how to evaluate speech models in a way that reflects real listeners, real accents, & real usage. This video explains how we ran this blind evaluation & what @JoshTalksLive is building in speech evaluations.

24,116 Aufrufe • vor 8 Monaten

Keine weiteren Inhalte verfügbar