Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

.Vogent just released Vogent-Turn-80M: an open-source, state-of-the-art turn-detection model for voice AI. It improves any voice AI stack by detecting when to wait and when to speak, reducing interruptions and making conversations more humanlike.

Y Combinator

1,655,509 subscribers

18,982 views • 1 year ago •via X (Twitter)

Health & Wellness Science & Technology Education

Anya Rossi• Live Now

Private livecam show

8 Comments

Arsh1 year ago

@vogentai This is so cool! @jraad27 I think you'll find this interesting.

Thierry1 year ago

@vogentai Congrats. Will give it a try for

Waqas | Ex-Google1 year ago

@vogentai 👀

AI PlanetX1 year ago

@vogentai Impressive innovation for voice AI! Excited to see more humanlike conversations.

Meddy 📎1 year ago

@Scobleizer @vogentai Trying now!

Trung Le1 year ago

@vogentai @livekit now has a competitor ( @vogentai's license is way better.

ENG-ZIYAD ALZAHRANI1 year ago

@Scobleizer @vogentai Nice

Jesse Ezell1 year ago

@vogentai @kwindla @davidzh seems nice

Related Videos

Vogent (Vogent) just launched extremely realistic voice agents. They rebuilt Sesame's CSM-1B model to produce ultra-humanlike voice in real time. Listen or call their AI Trump agent at 510-315-0014. Congrats on the launch, Jagath Vytheeswaran + Vignesh Varadarajan!

Vogent (Vogent) just launched extremely realistic voice agents. They rebuilt Sesame's CSM-1B model to produce ultra-humanlike voice in real time. Listen or call their AI Trump agent at 510-315-0014. Congrats on the launch, Jagath Vytheeswaran + Vignesh Varadarajan!

Y Combinator

45,876 views • 1 year ago

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

LiveKit

127,014 views • 1 year ago

Vogent (Vogent) just launched voice AI agents that design and improve themselves, learning from real failure cases and requiring no prompt engineering. Congrats on the launch, Jagath Vytheeswaran and Vignesh Varadarajan!

Vogent (Vogent) just launched voice AI agents that design and improve themselves, learning from real failure cases and requiring no prompt engineering. Congrats on the launch, Jagath Vytheeswaran and Vignesh Varadarajan!

Y Combinator

107,161 views • 1 year ago

Smart Turn v2: open source, native audio turn detection in 14 languages. New checkpoint of the open source, open data, open training code, semantic VAD model on Hugging Face, fal, and Pipecat AI. - 3x faster inference (12ms on an L40) - 14 languages (13 more than v1, which was english-only) - New synthetic data set `chirp_3_all` with ~163k audio samples - 99% accuracy on held out `human_5_all` test data Good turn detection is critical for voice agents. This model "understands" both semantic and audio patterns, and mitigates the voice AI trade-off between unwanted turn latency vs the agent interrupting people before they are finished speaking. Training scripts for both Modal and local training are in the repo. We want to make it as easy as possible to contribute to or customize this model! Here's a demo running the smart-turn model with default settings, aimed at generally hitting 400ms total turn detection time. You can tune things to be faster, too. You can help by contributing data, doing architecture expermints, or cleaning open source data! Keep reading ...

Smart Turn v2: open source, native audio turn detection in 14 languages. New checkpoint of the open source, open data, open training code, semantic VAD model on Hugging Face, fal, and Pipecat AI. - 3x faster inference (12ms on an L40) - 14 languages (13 more than v1, which was english-only) - New synthetic data set `chirp_3_all` with ~163k audio samples - 99% accuracy on held out `human_5_all` test data Good turn detection is critical for voice agents. This model "understands" both semantic and audio patterns, and mitigates the voice AI trade-off between unwanted turn latency vs the agent interrupting people before they are finished speaking. Training scripts for both Modal and local training are in the repo. We want to make it as easy as possible to contribute to or customize this model! Here's a demo running the smart-turn model with default settings, aimed at generally hitting 400ms total turn detection time. You can tune things to be faster, too. You can help by contributing data, doing architecture expermints, or cleaning open source data! Keep reading ...

kwindla

42,395 views • 1 year ago

Voice AI without the wait! ⏱️ Thanks to Hugging Face and Cerebras, developers can now use the Gemma 4 31B model as the brain for voice AI at ultra-fast inference speeds. Add it to a fully open-source, cascaded speech-to-speech stack that can be used to power existing voice apps! 🗣️

Voice AI without the wait! ⏱️ Thanks to Hugging Face and Cerebras, developers can now use the Gemma 4 31B model as the brain for voice AI at ultra-fast inference speeds. Add it to a fully open-source, cascaded speech-to-speech stack that can be used to power existing voice apps! 🗣️

Google Gemma

161,860 views • 3 months ago

.Vogent Voicelab is a high-performance API to run top open-source voice models, like Sesame CSM-1B, Dia, and more. Voicelab optimizes inference for each model so you can run them quickly and scalably— and they post-train select models to improve performance and reliability.

.Vogent Voicelab is a high-performance API to run top open-source voice models, like Sesame CSM-1B, Dia, and more. Voicelab optimizes inference for each model so you can run them quickly and scalably— and they post-train select models to improve performance and reliability.

Y Combinator

55,431 views • 1 year ago

🚨 BREAKING: NVIDIA just removed the biggest friction point in Voice AI They open-sourced PersonaPlex 7B, a real-time conversational model. It listens and speaks simultaneously to handle natural interruptions and overlaps. 100% Open Source.

🚨 BREAKING: NVIDIA just removed the biggest friction point in Voice AI They open-sourced PersonaPlex 7B, a real-time conversational model. It listens and speaks simultaneously to handle natural interruptions and overlaps. 100% Open Source.

How To Prompt

704,256 views • 7 months ago

Human conversation is one of the hardest problems in AI. Today, we're introducing Sparrow-2, our state-of-the-art, real-time conversational understanding model. It gives Tavus PALs something most voice AI still lacks: understanding what’s happening in a conversation and deciding what to do next- when to listen, wait, speak, or keep speaking.

Human conversation is one of the hardest problems in AI. Today, we're introducing Sparrow-2, our state-of-the-art, real-time conversational understanding model. It gives Tavus PALs something most voice AI still lacks: understanding what’s happening in a conversation and deciding what to do next- when to listen, wait, speak, or keep speaking.

Tavus

103,303 views • 2 months ago

Small AI voice models are getting interesting Small AI voice models are getting interesting. Not every text-to-speech application needs a massive model. I looked into Hojo-TTS-Light, an open-source TTS model family with 40M and 80M variants from HojoAI. Here's what makes it worth exploring. 🧵

Small AI voice models are getting interesting Small AI voice models are getting interesting. Not every text-to-speech application needs a massive model. I looked into Hojo-TTS-Light, an open-source TTS model family with 40M and 80M variants from HojoAI. Here's what makes it worth exploring. 🧵

ZARA

26,061 views • 11 days ago

Want to add offline voice AI to your ESP32? Start with ESP-Skainet. It is an open-source speech AI framework for building always-on voice assistants that run entirely on ESP32 devices. It combines WakeNet for wake-word detection, MultiNet for offline speech commands, and an Audio Front End with echo cancellation, noise suppression, and voice activity detection. ESP32-S3 is the recommended platform, with the latest models also targeting ESP32-P4.

Want to add offline voice AI to your ESP32? Start with ESP-Skainet. It is an open-source speech AI framework for building always-on voice assistants that run entirely on ESP32 devices. It combines WakeNet for wake-word detection, MultiNet for offline speech commands, and an Audio Front End with echo cancellation, noise suppression, and voice activity detection. ESP32-S3 is the recommended platform, with the latest models also targeting ESP32-P4.

Praveen Kumar Verma

15,209 views • 3 months ago

Today, we're introducing PlayHT2.0 - our new Conversational Text-to-Voice AI Model that's trained and built to generate humanlike conversations across languages with <1s latency. Sign up for beta access -

Today, we're introducing PlayHT2.0 - our new Conversational Text-to-Voice AI Model that's trained and built to generate humanlike conversations across languages with <1s latency. Sign up for beta access -

PlayAI

52,378 views • 3 years ago

Thrilled to see Amazon Web Services making a major contribution to the open source AI community with the launch of the Strands Agents, an open source AI agents SDK! The core of Strands is the simple agentic loop that connects the model and tools together, like the two strands of DNA. This model-driven approach to agent building eliminates the need for complex agent orchestration by embracing the capabilities of state-of-the-art models to plan, chain thoughts, call tools, and reflect. Providing open source tools and interoperability with open source protocols is an important part of our strategy to enable an agentic future. Can't wait to see what you build with Strands!

Thrilled to see Amazon Web Services making a major contribution to the open source AI community with the launch of the Strands Agents, an open source AI agents SDK! The core of Strands is the simple agentic loop that connects the model and tools together, like the two strands of DNA. This model-driven approach to agent building eliminates the need for complex agent orchestration by embracing the capabilities of state-of-the-art models to plan, chain thoughts, call tools, and reflect. Providing open source tools and interoperability with open source protocols is an important part of our strategy to enable an agentic future. Can't wait to see what you build with Strands!

Swami Sivasubramanian

32,314 views • 1 year ago

Introducing — a new conversational Voice AI platform and API for creating human-like voice agents and voice interfaces. Powered by our new Large Dialogue Model (LDM), Play LDM 1.0 makes the speech generation contextual; handles turn-taking, interruption, voice energy and emotion modulation for a natural, fluid, human conversations in real-time.

Introducing — a new conversational Voice AI platform and API for creating human-like voice agents and voice interfaces. Powered by our new Large Dialogue Model (LDM), Play LDM 1.0 makes the speech generation contextual; handles turn-taking, interruption, voice energy and emotion modulation for a natural, fluid, human conversations in real-time.

PlayAI

105,692 views • 2 years ago

Drive-thru voice AI that handles interruptions, noise, and natural conversations in Python. Ready? Let's build it ↓

Drive-thru voice AI that handles interruptions, noise, and natural conversations in Python. Ready? Let's build it ↓

Stream

614,977 views • 10 months ago

1. Voice activity detection To remove the need for pressing a button to start and stop recording of speech intention, we created a system for automatically detecting when a user is and isn’t trying to speak. This is a big step towards making communication for those with speech impairment completely hands-free.

1. Voice activity detection To remove the need for pressing a button to start and stop recording of speech intention, we created a system for automatically detecting when a user is and isn’t trying to speak. This is a big step towards making communication for those with speech impairment completely hands-free.

Neuralink

246,614 views • 4 days ago

AI researchers just developed a new Listening-While-Speaking Language Model (LSLM) The system can listen and speak simultaneously in real-time and respond to interruptions, a key feature of natural conversations

AI researchers just developed a new Listening-While-Speaking Language Model (LSLM) The system can listen and speak simultaneously in real-time and respond to interruptions, a key feature of natural conversations

Brett Adcock

19,059 views • 2 years ago

Clone any voice. Keep it local. Voice Clone Lab turns 5–15 minutes of audio into a voice model on your own GPU, cleanup, transcription, fine-tuning, and generation included. Open source: github/tetsuo-ai/voice_clone_lab

Clone any voice. Keep it local. Voice Clone Lab turns 5–15 minutes of audio into a voice model on your own GPU, cleanup, transcription, fine-tuning, and generation included. Open source: github/tetsuo-ai/voice_clone_lab

tetsuo

66,819 views • 3 months ago

NVIDIA just removed the biggest friction point in Voice AI 🤯 They've open-sourced PersonaPlex-7B, a full-duplex conversational model that can listen and speak at the same time. Instead of waiting for you to finish talking, it uses a dual-stream architecture to process and respond in real-time. 100% Open-Source and Free.

NVIDIA just removed the biggest friction point in Voice AI 🤯 They've open-sourced PersonaPlex-7B, a full-duplex conversational model that can listen and speak at the same time. Instead of waiting for you to finish talking, it uses a dual-stream architecture to process and respond in real-time. 100% Open-Source and Free.

Simplifying AI

279,234 views • 9 months ago