Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

You asked for speaker diarization, so we built it. Who split this 43-minute podcast by speaker in 4.9s on an M5 chip, including the moments two people talk at the same time. The recording never left the machine. Who is in our Swift, Kotlin and JavaScript SDKs, free up... show more

Desert Ant Labs

4,224 subscribers

47,217 views • 24 days ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

19 Comments

Gabriel Garrett24 days ago

Who’s on first?

Desert Ant Labs24 days ago

We dropped two new models this week. Read more:

Vadim24 days ago

practical application: healthcare dictation

Desert Ant Labs24 days ago

Doctor patient conversations are a perfect use case. Or anywhere you want meeting notes that stay private.

Kala24 days ago

calling the diarisation model Who is going to make the incident reports unbearable

Nathan Odle24 days ago

@drivelinekyle

R1ck van de Laar24 days ago

When do you expect that Who can run in the browser and in Node ?

Desert Ant Labs22 days ago

We're working on it.

Dhanji Bhagat24 days ago

43 min podcast in 4.9s on device is insane lol, and it even handles when 3 people talk at same time? that's the hard part

Lokutor24 days ago

4.9s for a 43-minute podcast on-device is a strong result, especially with overlap handled locally. Keeping diarization off the cloud makes the whole voice stack easier to trust. The Swift/Kotlin/JS SDK spread is the practical part too.

R1ck van de Laar24 days ago

@grok how is support for Dutch

Thijs Sondag24 days ago

Epic! Will try it out. Tested performance on iPhone yes?

Desert Ant Labs24 days ago

Yes. On an iPhone 16 Pro, Who labels 5 minutes of audio in about half a second, around 540x realtime. It runs on Android too, at about 55x on a Pixel 11. Let us know how it goes!

Miru23 days ago

@ThijsSondag But does it do realtime ?

Desert Ant Labs23 days ago

Not yet. Who is designed to label a finished recording rather than a live stream. What are you working on? Live speaker labeling is an interesting challenge.

Harrison24 days ago

if someone's entire contribution is a half-second 'no', could their speaker label stay unknown? the fallback on the model page would attach it to the nearest speaker. i'd rather leave the objection unattributed.

Desert Ant Labs24 days ago

A short “no” usually gets its own speaker number. The exception is a long recording where someone says only a half-second “no” in a whole five-minute stretch. That goes to the nearest speaker, and there’s no setting yet to leave it unlabeled.

JΞr₳h24 days ago

43 minutes split in 4.9s on device is nuts. keeping the audio local is a big deal for podcasters

Marko24 days ago

4.9 s on 43 minutes is ~526x real time, so speed is settled. The number that decides if this is usable is the error rate on the overlapping stretches: keeping the moments two people talk over each other is not the same as labelling them right.

Related Videos

microsoft just released a tool that transcribes a full hour of audio at once, tracking who spoke and when. it's called vibevoice the 7b model processes the full recording in one shot instead of chunking it, so speaker identity and context never break across segments → full speaker diarization built in, not bolted on → structured timestamps for every speaker turn → supports 50+ languages runs locally, no api costs.

microsoft just released a tool that transcribes a full hour of audio at once, tracking who spoke and when. it's called vibevoice the 7b model processes the full recording in one shot instead of chunking it, so speaker identity and context never break across segments → full speaker diarization built in, not bolted on → structured timestamps for every speaker turn → supports 50+ languages runs locally, no api costs.

Oliver

53,779 views • 3 months ago

Rep. Tom Emmer is running for Speaker. In 2015 he was on our show and Tane Danger asked him about the Speaker fight happening then. He told this story (in a solid Rep. Trey Gowdy impersonation) with the moral, “Who would ever want to be Speaker?”

Rep. Tom Emmer is running for Speaker. In 2015 he was on our show and Tane Danger asked him about the Speaker fight happening then. He told this story (in a solid Rep. Trey Gowdy impersonation) with the moral, “Who would ever want to be Speaker?”

TheaterPublicPolicy

21,845 views • 3 years ago

imagine having a Speaker of the House who is actually a good speaker for the first time in 30 years Congresswoman Ayanna Pressley we need you to run

imagine having a Speaker of the House who is actually a good speaker for the first time in 30 years Congresswoman Ayanna Pressley we need you to run

ostonox

19,125 views • 3 days ago

We built our own lane, on our own terms and 300,000 people chose to be part of it. Our community is our compass. We won’t change who we are for people who were never the audience. If it’s not for you don’t tune in.

We built our own lane, on our own terms and 300,000 people chose to be part of it. Our community is our compass. We won’t change who we are for people who were never the audience. If it’s not for you don’t tune in.

Sho Up 🏴‍☠️

510,752 views • 1 month ago

When a motion for the removal of the Speaker is taken up, Article 96 makes it very clear that the Speaker cannot sit in the Chair. The actions of the Speaker also cannot be admitted at that point of time. You (Jagdambika Pal) have been appointed by the Speaker of the Lok Sabha. You are sitting there as a member of the panel of chairpersons appointed by the Speaker of the Lok Sabha. Article 93 states: “The House of the People shall, as soon as may be, choose two members of the House to be respectively Speaker and Deputy Speaker thereof and, so often as the office of the Speaker or Deputy Speaker becomes vacant, the House shall choose another member to be Speaker or Deputy Speaker, as the case may be.” Immediately after the election, the Speaker and Deputy Speaker are supposed to be elected. For the last seven years, you have not elected a Deputy Speaker. You have created a constitutional vacuum. This question is very clear. Article 96 is very clear. 👉The House has to select a person to sit in the Chair. : Congress General Secretary (Org.) K C Venugopal ji in Lok Sabha

When a motion for the removal of the Speaker is taken up, Article 96 makes it very clear that the Speaker cannot sit in the Chair. The actions of the Speaker also cannot be admitted at that point of time. You (Jagdambika Pal) have been appointed by the Speaker of the Lok Sabha. You are sitting there as a member of the panel of chairpersons appointed by the Speaker of the Lok Sabha. Article 93 states: “The House of the People shall, as soon as may be, choose two members of the House to be respectively Speaker and Deputy Speaker thereof and, so often as the office of the Speaker or Deputy Speaker becomes vacant, the House shall choose another member to be Speaker or Deputy Speaker, as the case may be.” Immediately after the election, the Speaker and Deputy Speaker are supposed to be elected. For the last seven years, you have not elected a Deputy Speaker. You have created a constitutional vacuum. This question is very clear. Article 96 is very clear. 👉The House has to select a person to sit in the Chair. : Congress General Secretary (Org.) K C Venugopal ji in Lok Sabha

Congress

112,838 views • 8 months ago

🚨I will continue to vote for our duly-elected Republican Speaker Kevin McCarthy, who should have never been removed! In the meantime, we must empower Speaker Pro-Tempore Patrick McHenry to get on with the work of the American people. Thank you, Trey Gowdy, I agree with you too!

🚨I will continue to vote for our duly-elected Republican Speaker Kevin McCarthy, who should have never been removed! In the meantime, we must empower Speaker Pro-Tempore Patrick McHenry to get on with the work of the American people. Thank you, Trey Gowdy, I agree with you too!

Rep. Carlos A. Gimenez

103,415 views • 3 years ago

We should get to electing a new, more conservative and more trustworthy Speaker immediately. I'm calling on Patrick McHenry, who is currently the Speaker Pro Tem, to reconsider the decision that he just made to send everyone in Congress home for a week. This is not the time to go home for a week. We should stay and elect a new Speaker.

We should get to electing a new, more conservative and more trustworthy Speaker immediately. I'm calling on Patrick McHenry, who is currently the Speaker Pro Tem, to reconsider the decision that he just made to send everyone in Congress home for a week. This is not the time to go home for a week. We should stay and elect a new Speaker.

Former Congressman Matt Gaetz

2,256,451 views • 3 years ago

NVIDIA released a fast speaker-labeling model today: Nemotron 3 Diarization. Like all of the Nemotron family from @NVIDIA_AI, this is an open weights model, so you can grab the weights from Hugging Face and use the model in the cloud or on your local device. Nemotron 3 Diarization is a useful building block for voice agents and realtime AI applications that need to handle speech from multiple people at the same time. It handles overlapping speech well, can identify up to eight speakers, and pairs with any transcription model you're using in your multi-modal pipelines. Jon Taylor built a demo (code below) and recorded a technical deep-dive showing how to use this diarization model together with a speech-to-text model. Jon's code also uses Jev for super-accurate confirmation of critical voice input in a noisy/multi-speaker environment.

NVIDIA released a fast speaker-labeling model today: Nemotron 3 Diarization. Like all of the Nemotron family from @NVIDIA_AI, this is an open weights model, so you can grab the weights from Hugging Face and use the model in the cloud or on your local device. Nemotron 3 Diarization is a useful building block for voice agents and realtime AI applications that need to handle speech from multiple people at the same time. It handles overlapping speech well, can identify up to eight speakers, and pairs with any transcription model you're using in your multi-modal pipelines. Jon Taylor built a demo (code below) and recorded a technical deep-dive showing how to use this diarization model together with a speech-to-text model. Jon's code also uses Jev for super-accurate confirmation of critical voice input in a noisy/multi-speaker environment.

kwindla

12,729 views • 1 month ago

The same people who say they’re going to sit out the general election are never the same people who were active in the primaries. That’s where the fight is supposed to take place. The general election is about unity, not the perfect candidate. When we say unity, we mean fighting the Left. That’s it. If someone claims to be in our party, in our wing, conservative, MAGA, America First, Republican, but they aren't fighting the Left, that's the standard. They are not to be included.

The same people who say they’re going to sit out the general election are never the same people who were active in the primaries. That’s where the fight is supposed to take place. The general election is about unity, not the perfect candidate. When we say unity, we mean fighting the Left. That’s it. If someone claims to be in our party, in our wing, conservative, MAGA, America First, Republican, but they aren't fighting the Left, that's the standard. They are not to be included.

Steven Crowder

96,551 views • 18 days ago

Introducing Diarization Bench, Voice Arena's benchmark for who spoke when. The metric we measure is diarization error rate, or DER. It measures how much of a conversation a model attaches to the wrong person. Every second it misses, invents, or gives to another speaker counts against it. Here's what we did. 139 conversations recorded for this board, in rooms and on calls, with people interrupting each other throughout. 12 systems scored under a single goal: testing models in the real-world conditions they are actually deployed in. NVIDIA's Nemotron 3 leads at 14.72%. 🧵

Introducing Diarization Bench, Voice Arena's benchmark for who spoke when. The metric we measure is diarization error rate, or DER. It measures how much of a conversation a model attaches to the wrong person. Every second it misses, invents, or gives to another speaker counts against it. Here's what we did. 139 conversations recorded for this board, in rooms and on calls, with people interrupting each other throughout. 12 systems scored under a single goal: testing models in the real-world conditions they are actually deployed in. NVIDIA's Nemotron 3 leads at 14.72%. 🧵

Shobhit Banga

647,972 views • 1 month ago

Hasan Piker rages at chat once again, this time mocking them for wanting a more and more radical Speaker of the House... "I think it should be Yahya Sinwar. I think we should appoint Yahya Sinwar to be the speaker of the house..Why isn't someone who is a real anti imperialist who will become the speaker of the house? I don't know guys. Maybe because we live in the fucking real world"

Hasan Piker rages at chat once again, this time mocking them for wanting a more and more radical Speaker of the House... "I think it should be Yahya Sinwar. I think we should appoint Yahya Sinwar to be the speaker of the house..Why isn't someone who is a real anti imperialist who will become the speaker of the house? I don't know guys. Maybe because we live in the fucking real world"

PikerWatch

244,000 views • 4 days ago

The world didn’t just show up like this. It was built. By people who wake up before sunrise. People who make things. People who take risks. People who refuse to stand still. Power for the Free is our story about the builders shaping America’s future. This is for everyone creating what comes next.

The world didn’t just show up like this. It was built. By people who wake up before sunrise. People who make things. People who take risks. People who refuse to stand still. Power for the Free is our story about the builders shaping America’s future. This is for everyone creating what comes next.

T1 Energy

1,715,115 views • 4 months ago

Mike Johnson has barely been Speaker for 6 months and led us to a complete catastrophe by passing Chuck Schumer omnibus wishlist. He failed to keep his promises, including using the appropriations process to secure our border! We need a Speaker who will lead the Republican Party and leverage our majority in the House.

Mike Johnson has barely been Speaker for 6 months and led us to a complete catastrophe by passing Chuck Schumer omnibus wishlist. He failed to keep his promises, including using the appropriations process to secure our border! We need a Speaker who will lead the Republican Party and leverage our majority in the House.

Former Congresswoman Marjorie Taylor Greene🇺🇸

497,807 views • 2 years ago

Don’t let the chip on your shoulder get so heavy that you can’t lift your hands to praise God. Sometimes we become so focused on the negativity that fuels our drive to succeed that we forget to celebrate the One—and the people—who made it possible. Embrace the love of those who support you, and lead with gratitude for every achievement. If your only motivation is proving the doubters wrong, you may reach success but never find peace in it. Pivot Podcast #ThePivot #RCconfessionals #PivotalPoints

Don’t let the chip on your shoulder get so heavy that you can’t lift your hands to praise God. Sometimes we become so focused on the negativity that fuels our drive to succeed that we forget to celebrate the One—and the people—who made it possible. Embrace the love of those who support you, and lead with gratitude for every achievement. If your only motivation is proving the doubters wrong, you may reach success but never find peace in it. Pivot Podcast #ThePivot #RCconfessionals #PivotalPoints

Ryan Clark

32,352 views • 3 months ago

236 years ago, the Founders signed our Constitution, beginning our great American experiment. As Speaker, it is my duty to use every tool in the Constitution—including the power of the purse and oversight of the Executive Branch—to ensure Congress stands up for We the People.

236 years ago, the Founders signed our Constitution, beginning our great American experiment. As Speaker, it is my duty to use every tool in the Constitution—including the power of the purse and oversight of the Executive Branch—to ensure Congress stands up for We the People.

Kevin McCarthy

331,243 views • 3 years ago

Not everyone can listen to an 80-minute podcast, so I narrated this animated short so that people can understand our reporting on the The New York Times in less than two minutes. This story is not going away, and we will continue to follow it.

Not everyone can listen to an 80-minute podcast, so I narrated this animated short so that people can understand our reporting on the The New York Times in less than two minutes. This story is not going away, and we will continue to follow it.

Imara Jones

31,253 views • 3 years ago

PLU says it does not support Anita Among and Thomas Tayebwa for the Speaker and Deputy Speaker positions in the upcoming parliamentary speaker race. The CDF has asked NRM Members of Parliament to wait for President Museveni to come up with the names of the two candidates. PLU had earlier endorsed the two candidates. Note : the announcement has been made by Among's friend Kabanda.

PLU says it does not support Anita Among and Thomas Tayebwa for the Speaker and Deputy Speaker positions in the upcoming parliamentary speaker race. The CDF has asked NRM Members of Parliament to wait for President Museveni to come up with the names of the two candidates. PLU had earlier endorsed the two candidates. Note : the announcement has been made by Among's friend Kabanda.

Sudhir Byaruhanga

88,383 views • 6 months ago

#BlackAmerica, we see it. We feel it. We tired of it. 🇺🇸🔥 This video right here is the perfect example. You can walk in rocking our style, listening to our music, taking jobs and opportunities rooted in Black American culture, being fully influenced by everything we built… then turn around and talk crazy like directors prefer UK actors because “Americans just want to be famous.” That’s the same energy as the people who steal our culture and then talk down on us. 👀 We built this culture through real struggle. Our sons getting innocently shot in the street, systems failing us, generations of fighting just to exist — that’s the foundation. You don’t get to benefit from the culture and then disrespect the people who created it. Black American culture is not up for grabs. Respect it or keep it moving. We standing on that. 💯 What’s y’all take on this? Drop your thoughts below. #BlackAmerican #RespectTheCulture #KushNation🔴🔱⚫️ #NoMoreErasure 👑

#BlackAmerica, we see it. We feel it. We tired of it. 🇺🇸🔥 This video right here is the perfect example. You can walk in rocking our style, listening to our music, taking jobs and opportunities rooted in Black American culture, being fully influenced by everything we built… then turn around and talk crazy like directors prefer UK actors because “Americans just want to be famous.” That’s the same energy as the people who steal our culture and then talk down on us. 👀 We built this culture through real struggle. Our sons getting innocently shot in the street, systems failing us, generations of fighting just to exist — that’s the foundation. You don’t get to benefit from the culture and then disrespect the people who created it. Black American culture is not up for grabs. Respect it or keep it moving. We standing on that. 💯 What’s y’all take on this? Drop your thoughts below. #BlackAmerican #RespectTheCulture #KushNation🔴🔱⚫️ #NoMoreErasure 👑

MF KUSH 🔴🔱⚫️

17,275 views • 7 days ago

This morning we submitted our final recommendations on the 2023 Boundary Review to the Speaker of the House of Commons. Thanks to everyone who contributed and participated in the review.

This morning we submitted our final recommendations on the 2023 Boundary Review to the Speaker of the House of Commons. Thanks to everyone who contributed and participated in the review.

Boundary Commission for England

33,279 views • 3 years ago