I'm annoyed with voice AIs that interrupt me before I'm done speaking 😠 So here's a demo (sound on! 🔉) where it only speaks when I'm done talking, and stops when I interrupt it. It's crazy how much more natural this feels! Deets below 👇🧵

Super simple recipe: 1) Speech transcript is formatted like a screenplay w user and AI as speakers 2) LLM continuously predicts the future of the transcript 3) if LLM predicts user will keep speaking, it waits 4) if LLM predicts AI will start speaking, it speaks prediction aloud

Core ingredients: ✅ real-time speech recognition w diarization ✅ a fast LLM w completion API ✅ interruptible speech synthesis

woah amazing how little latency there is looking to build / improve my own systems soon, just wondering if this is what you're also using / recommend: - faster-whisper / distil-whisper for stt - pyannote for diarization - quantized 7B model for fast llm - elevenlabs turbo or equivalent tts with streaming api - silero vad

I think all of that should work! Fwiw my stack is v vanilla haha: * STT: deepgram streaming API w diarization enabled * LLM: GPT-3.5-turbo-instruct * TTS: WebSpeech API * no vad 🙊

This is great!

this is such an improvement. my 5 year old talks to chatGPT thru voice and gets so frustrated when it keeps talking over her, or starts responding before she's done talking, and doesn't stop to listen

Fascinated by your demo, I'm the maintainer of Vocode Python at Any plans to open-source this? If so, I'd love to discuss integrating it into our library, widely used by developers for voice LLMS apps.

Is there a repo for this? It’s so nice 😍

Very nice! Reminds me of some of the ideas I posted last year!

actually yes, totally same wavelength! Great minds think alike