Drop 1/15: Introducing Joy, a dual-channel conversational dataset. Real, unscripted conversations between two people, each on their own track. Built to teach voice models how people actually talk: when to listen, when to jump in, when to say “hmm”. 100,000 hours. 50,000 speakers. 10 languages.

So, what difference does this kind of data make? Here’s PersonaPlex, fine-tuned on just a fraction of Joy. It backchannels while you’re speaking, takes the floor at natural transition points, and can hold a pause without giving up its turn. Worth looking at why this matters: what do these models usually learn from in the first place?

Large-scale dual-channel conversational data is still hard to come by. The usual alternative? Clean, synthetic dialogue: scripts written by LLMs and then rendered using TTS. But real conversations don’t work like scripts. People overlap, interrupt each other, and make little sounds to show they’re listening. Joy keeps all of that in. Across Joy, that works out to around 300 backchannels/hr, 350 overlaps/hr, and 90 interruptions/hr.

Here’s 3.5 minutes of unscripted US English from Joy, with one speaker on each channel. You can see the backchannels, overlaps, and pauses marked as they happen. No script. Just the conversation as it happened.

And it’s not just English. Here’s a sample in Hindi. Same setup: two speakers, two channels, and nothing cleaned up. Joy covers 10 languages, all recorded the same way.

A closer look at how the 100,000 hours are distributed across speakers. There are 50,000 speakers in total, and no single speaker accounts for more than 0.02% of the hours. That’s a lot of different voices, without any one speaker dominating the dataset.

For context, natural dual-channel conversational corpora have, until now, been limited to under 2,000 hours. Joy brings that to 100,000 hours. And that’s just where it starts. v1 - Oct 26 - 100k v2 - Mar 27 - 1M v3 - Mar 28 - 10M

The backchannels while the other person is still talking are what got me.

Shobhit, wow I love this! Great work!

Separate tracks per speaker is the underrated detail here.

Is there an eval set alongside the training data?

Hard to fake real turn-taking. This data doesn't have to.

What happens on a noisy line? Would be good to see that covered.

Support lines are going to feel very different in a year or two.

Is there a write-up on how the data was collected?

The size is great but the separate channels are what make it usable.

A bit uneasy about how natural agents will get with data like this.

Is the sample rate the same across all languages?

Real people don't wait for silence. Good to see data that shows it.

This would make customer support calls a lot less painful.

Real interruption timing is the thing synthetic data never gets right.

A before/after on a model trained with and without this data would be great.

Only part of the data was used for the demo? The full set must go further.

Not sure I'd catch a well-trained agent on a short call anymore.

Big dataset, but I’m more curious about the quality checks. Anyone know how it was cleaned?

Curious about the average call length. Longer calls would be more useful for this.

First duplex dataset I’ve seen where hours aren’t the limiting factor.

Would love the per-language breakdown.

Timing is most of conversation, and this data has it.

50,000 speakers is the stat I'd look at first.

Would like to hear longer calls from the dataset.

How were speakers recruited? Curious about the process at this scale.

How do the two clips in different languages compare on overlap rate?

That comparison chart is pretty striking.

Didn't realise how much people overlap on normal calls until these clips.

Would love to know the average number of overlaps per minute.

How long did recording take overall?

This is the data problem a lot of us have been complaining about for years.

Speaker count matters more than hours for generalising, in my experience.

Turn-taking is clearly a data problem more than a model problem.

Breadth over a few heavy speakers, good call.