Shobhit Banga's banner
Shobhit Banga's profile picture

Shobhit Banga

@shobhitbanga • 3,391 subscribers

Scorekeeper for voice AI. Building independent, real-world evals to help labs build the best voice models. Co-founder @voicearena_ai | prev @JoshTalksAI

Shorts

Every language in Monsoon V1, in one spin of the globe. 50 languages. 23 countries. Spontaneous speech, not read prompts. If your ASR works in English and falls apart everywhere else, this is the data it was missing.

Every language in Monsoon V1, in one spin of the globe. 50 languages. 23 countries. Spontaneous speech, not read prompts. If your ASR works in English and falls apart everywhere else, this is the data it was missing.

173,286 views

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

Introducing Monsoon ASR ⚡️ Voice Arena Speech recognition does not have a model problem anymore. It has a data problem. The best ASR systems are approaching human-level performance in English. But move into the long tail of the world's languages, especially real, conversational speech and error rates can still be 5-10X higher. Today, we're releasing Monsoon ASR: a new generation of training data built specifically to close that gap. 50 languages. 100,000+ hours. Dense spontaneous speech. And one goal: Single-digit WER across the world's languages.🧵
4:00
shobhitbanga's profile picture

Introducing Monsoon ASR ⚡️ Voice Arena Speech recognition does not have a model problem anymore. It has a data problem. The best ASR systems are approaching human-level performance in English. But move into the long tail of the world's languages, especially real, conversational speech and error rates can still be 5-10X higher. Today, we're releasing Monsoon ASR: a new generation of training data built specifically to close that gap. 50 languages. 100,000+ hours. Dense spontaneous speech. And one goal: Single-digit WER across the world's languages.🧵

Shobhit Banga

648,443 views • 28 days ago

Introducing Diarization Bench, Voice Arena's benchmark for who spoke when. The metric we measure is diarization error rate, or DER. It measures how much of a conversation a model attaches to the wrong person. Every second it misses, invents, or gives to another speaker counts against it. Here's what we did. 139 conversations recorded for this board, in rooms and on calls, with people interrupting each other throughout. 12 systems scored under a single goal: testing models in the real-world conditions they are actually deployed in. NVIDIA's Nemotron 3 leads at 14.72%. 🧵
4:16
shobhitbanga's profile picture

Introducing Diarization Bench, Voice Arena's benchmark for who spoke when. The metric we measure is diarization error rate, or DER. It measures how much of a conversation a model attaches to the wrong person. Every second it misses, invents, or gives to another speaker counts against it. Here's what we did. 139 conversations recorded for this board, in rooms and on calls, with people interrupting each other throughout. 12 systems scored under a single goal: testing models in the real-world conditions they are actually deployed in. NVIDIA's Nemotron 3 leads at 14.72%. 🧵

Shobhit Banga

647,052 views • 1 month ago

Introducing Jarvis Bench v0.5, Voice Arena's conversational agent benchmark. We've been obsessed with one question at VoiceArena: why do voice agent demos sound incredible, benchmarks say models are near-perfect, and yet you probably didn't have a single real conversation with a voice agent in the last 24 hours? Here's what's different about the Jarvis Bench. Real humans have live conversations with voice agents. A second group of humans blind-votes pairwise on two questions: which sounded more human, which got the job done. And one of the "agents" on the leaderboard is a human.
4:15
shobhitbanga's profile picture

Introducing Jarvis Bench v0.5, Voice Arena's conversational agent benchmark. We've been obsessed with one question at VoiceArena: why do voice agent demos sound incredible, benchmarks say models are near-perfect, and yet you probably didn't have a single real conversation with a voice agent in the last 24 hours? Here's what's different about the Jarvis Bench. Real humans have live conversations with voice agents. A second group of humans blind-votes pairwise on two questions: which sounded more human, which got the job done. And one of the "agents" on the leaderboard is a human.

Shobhit Banga

573,895 views • 1 month ago

We’ve spent years collecting & annotating speech data. What became equally important was learning how to evaluate speech models in a way that reflects real listeners, real accents, & real usage. This video explains how we ran this blind evaluation & what @JoshTalksLive is building in speech evaluations.
1:54
shobhitbanga's profile picture

We’ve spent years collecting & annotating speech data. What became equally important was learning how to evaluate speech models in a way that reflects real listeners, real accents, & real usage. This video explains how we ran this blind evaluation & what @JoshTalksLive is building in speech evaluations.

Shobhit Banga

24,116 views • 8 months ago

No more content to load