Loading video...

Video Failed to Load

Go Home

With Invincible Voice, we help people living with ALS communicate more easily. Encountering Olivier Goy, an entrepreneur who lives with ALS and relentlessly fights to help all patients, made it obvious that our cutting-edge voice AI should help. We turned our Unmute voice-wrapper into a new system that 1/...

kyutai_labs's profile picture

kyutai

27,828 subscribers

11,134 views • 8 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

1 Comments

Francisco Cordoba's profile picture
Francisco Cordoba8 months ago

I love your company! amazing tech! Been doing some experiments!

Related Videos

Excited to introduce Fish Speech 1.4 - now open-source and more powerful than ever! 🎉 Our mission is to make cutting-edge voice tech accessible to everyone. What's new: - Trained on 700k hours of multilingual data (up from 200k) - Now supports 8 languages: English, Chinese, German, Japanese, French, Spanish, Korean, and Arabic - Fully open-source, empowering developers and researchers worldwide Key features: - Lightning-fast TTS with ultra-low latency - Instant voice cloning - Self-host or use our cloud service - Simple, flat-rate pricing Try it out: - Playground: - GitHub: - HuggingFace Model: - Demo: - Product Hunt: We can't wait to see what you'll create with Fish Audio. Happy voice building! 🎧🐠
1:23

Excited to introduce Fish Speech 1.4 - now open-source and more powerful than ever! 🎉 Our mission is to make cutting-edge voice tech accessible to everyone. What's new: - Trained on 700k hours of multilingual data (up from 200k) - Now supports 8 languages: English, Chinese, German, Japanese, French, Spanish, Korean, and Arabic - Fully open-source, empowering developers and researchers worldwide Key features: - Lightning-fast TTS with ultra-low latency - Instant voice cloning - Self-host or use our cloud service - Simple, flat-rate pricing Try it out: - Playground: - GitHub: - HuggingFace Model: - Demo: - Product Hunt: We can't wait to see what you'll create with Fish Audio. Happy voice building! 🎧🐠

Fish Audio

149,977 views • 2 years ago

We're launching the ElevenLabs Impact Program to help 1 million people reclaim their voice. We’re partnering with nonprofits to provide free licenses to enhance accessibility, advance education, and improve shared cultural experiences. Our first initiative, made possible through partnership with Bridging Voice and The Scott-Morgan Foundation, is aimed at helping everyone with ALS / MND preserve their voice through free access to our technology to create personalized digital voices that closely match their natural speech. If this can benefit you or a loved one, learn more here: If your nonprofit can benefit from AI Audio, get in touch:
0:58

We're launching the ElevenLabs Impact Program to help 1 million people reclaim their voice. We’re partnering with nonprofits to provide free licenses to enhance accessibility, advance education, and improve shared cultural experiences. Our first initiative, made possible through partnership with Bridging Voice and The Scott-Morgan Foundation, is aimed at helping everyone with ALS / MND preserve their voice through free access to our technology to create personalized digital voices that closely match their natural speech. If this can benefit you or a loved one, learn more here: If your nonprofit can benefit from AI Audio, get in touch:

ElevenLabs

109,635 views • 2 years ago

Voice agents still don’t understand who’s speaking to them. That’s a huge gap compared with humans, hidden by all the “phone-call” demos. But that changes today! NVIDIA is open-sourcing Nemotron 3 Diarization: a model that can reliably track speakers in live conversations, under a commercial-friendly license! In my tests, the quality is really good with one-second speech chunks. So we can use it for voice agents! I tested it with Reachy Mini and speech-to-speech running on a DGX Spark. It’s super fun to see the robot notice a new voice, ask for a name, and remember it. The model has day-zero integration with Transformers! Kudos to the NVIDIA team for shipping useful tools for the whole community!
0:33

Voice agents still don’t understand who’s speaking to them. That’s a huge gap compared with humans, hidden by all the “phone-call” demos. But that changes today! NVIDIA is open-sourcing Nemotron 3 Diarization: a model that can reliably track speakers in live conversations, under a commercial-friendly license! In my tests, the quality is really good with one-second speech chunks. So we can use it for voice agents! I tested it with Reachy Mini and speech-to-speech running on a DGX Spark. It’s super fun to see the robot notice a new voice, ask for a name, and remember it. The model has day-zero integration with Transformers! Kudos to the NVIDIA team for shipping useful tools for the whole community!

Andi Marafioti

78,328 views • 15 days ago

Excited about the launch of Amazon Nova Sonic, our new speech-to-speech model that helps make AI voice applications feel remarkably natural. It's designed to understand not just what people say, but how they say it – working with tone, style, and conversation flow including pauses and interruptions. Nova Sonic delivers speech understanding and generation through a single, unified model, making it easier for builders to develop voice applications that maintain important context and nuance for customer service, AI agents, and other use cases across industries. It’s available in Amazon Bedrock now. Look forward to seeing what teams build with Nova Sonic!
1:25

Excited about the launch of Amazon Nova Sonic, our new speech-to-speech model that helps make AI voice applications feel remarkably natural. It's designed to understand not just what people say, but how they say it – working with tone, style, and conversation flow including pauses and interruptions. Nova Sonic delivers speech understanding and generation through a single, unified model, making it easier for builders to develop voice applications that maintain important context and nuance for customer service, AI agents, and other use cases across industries. It’s available in Amazon Bedrock now. Look forward to seeing what teams build with Nova Sonic!

Andy Jassy

155,926 views • 1 year ago

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images! It sees, understands, and talks about images — naturally, and out loud. Voice interaction with a compact model endowed with visual understanding opens up new applications, from audio description for the visual impaired to visual access to information. Try it out 👉 Blog post 👉
0:37

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images! It sees, understands, and talks about images — naturally, and out loud. Voice interaction with a compact model endowed with visual understanding opens up new applications, from audio description for the visual impaired to visual access to information. Try it out 👉 Blog post 👉

kyutai

48,003 views • 1 year ago

We just shipped something nobody else has. Our STT API doesn't just transcribe, it returns a full voice profile in the same response: emotion, accent, age, pitch, style, each with a confidence score, all in one call. When you feed that into our Realtime API, the LLM decides not just what to say but how to say it. It adds steering tags that our TTS-2 model picks up and uses to guide how it speaks. So, if a user sounds frustrated, the voice responds with empathy. If they're excited, it matches that energy. Here's our very own Clint McLean walking you through how the full pipeline works:
1:24

We just shipped something nobody else has. Our STT API doesn't just transcribe, it returns a full voice profile in the same response: emotion, accent, age, pitch, style, each with a confidence score, all in one call. When you feed that into our Realtime API, the LLM decides not just what to say but how to say it. It adds steering tags that our TTS-2 model picks up and uses to guide how it speaks. So, if a user sounds frustrated, the voice responds with empathy. If they're excited, it matches that energy. Here's our very own Clint McLean walking you through how the full pipeline works:

Inworld AI

250,703 views • 4 months ago

Today we’re excited to introduce Bland TTS, the first voice AI to cross the uncanny valley. Several months ago, our team solved one-shot style transfer of human speech. That means, from a single, brief MP3, you can clone any voice or remix another clone’s style (tone, cadence, or pronunciations, etc.). With Bland TTS: • Creatives can turn text into realistic sound effects or AI voice tracks with precise control over style and emotion. • Developers can plug in our TTS API to build voice apps fully customized to their liking. • Enterprises can launch AI-driven support lines that sound so natural — customers will save you as a contact (try talking to it on our site). 🧵Check out the thread below or visit our website to get started for free
2:30

Today we’re excited to introduce Bland TTS, the first voice AI to cross the uncanny valley. Several months ago, our team solved one-shot style transfer of human speech. That means, from a single, brief MP3, you can clone any voice or remix another clone’s style (tone, cadence, or pronunciations, etc.). With Bland TTS: • Creatives can turn text into realistic sound effects or AI voice tracks with precise control over style and emotion. • Developers can plug in our TTS API to build voice apps fully customized to their liking. • Enterprises can launch AI-driven support lines that sound so natural — customers will save you as a contact (try talking to it on our site). 🧵Check out the thread below or visit our website to get started for free

Bland

46,014,197 views • 1 year ago

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.
1:21

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.

Rohan Paul

15,672 views • 6 months ago

Introducing Multilingual Voices We partnered with Baklavastory, our neighbor in the Mission, to show what it sounds like when a business's character and warmth stay consistent, no matter who calls or what language they speak. It's easy to translate speech, but much harder to preserve a single identity across languages. Every language has different rhythms, tones, and emphasis. When you change the language, identity tends to get lost in that shift, and your voice ends up sounding like someone else. With Multilingual Voices, pick a voice and keep one brand identity in every market you serve:
1:32

Introducing Multilingual Voices We partnered with Baklavastory, our neighbor in the Mission, to show what it sounds like when a business's character and warmth stay consistent, no matter who calls or what language they speak. It's easy to translate speech, but much harder to preserve a single identity across languages. Every language has different rhythms, tones, and emphasis. When you change the language, identity tends to get lost in that shift, and your voice ends up sounding like someone else. With Multilingual Voices, pick a voice and keep one brand identity in every market you serve:

Karan Goel

5,400,694 views • 21 days ago

Today, we’re making Sarvam Content Studio available to everyone. Content Studio brings everything you need to create, edit and transform voice and video content across languages into one workspace. You can turn a script into natural speech, clone a voice from a short recording, or dub a video into multiple languages while preserving the original speaker’s voice, tone and rhythm. You also have far more control over what happens after generation, with the ability to review every line, create subtitles, make precise edits and keep each version consistent before it goes out. We built Content Studio to help teams create content and effortlessly transform it for wider audiences, languages, and formats. Start creating:
1:27

Today, we’re making Sarvam Content Studio available to everyone. Content Studio brings everything you need to create, edit and transform voice and video content across languages into one workspace. You can turn a script into natural speech, clone a voice from a short recording, or dub a video into multiple languages while preserving the original speaker’s voice, tone and rhythm. You also have far more control over what happens after generation, with the ability to review every line, create subtitles, make precise edits and keep each version consistent before it goes out. We built Content Studio to help teams create content and effortlessly transform it for wider audiences, languages, and formats. Start creating:

Sarvam

44,689 views • 1 month ago

Meet the new Soniox voice library. Soniox Text-to-Speech now includes more than 200 built-in voices, carefully curated for different applications, styles, accents, ages, and voice characteristics. Every voice works across 60+ languages, so you can choose the voice that fits your product and use that same voice globally while preserving its identity and character. You can now browse and filter voices by: • Gender • Age • Accent • Use case, such as conversational, narration, or social media • Style, such as warm, confident, friendly, calm, energetic, or deep This makes it much easier to find a voice for a specific application, for example: • Conversational + warm + middle-aged • Narration + deep + confident • Social media + young + energetic • Educational + calm + friendly We manually selected, reviewed, and tagged the voices so the metadata reflects how the voices actually sound and where they work well. We also redesigned the TTS Playground to make it easier to browse, filter, compare, and test voices with your own text and languages. Learn more:
0:42

Meet the new Soniox voice library. Soniox Text-to-Speech now includes more than 200 built-in voices, carefully curated for different applications, styles, accents, ages, and voice characteristics. Every voice works across 60+ languages, so you can choose the voice that fits your product and use that same voice globally while preserving its identity and character. You can now browse and filter voices by: • Gender • Age • Accent • Use case, such as conversational, narration, or social media • Style, such as warm, confident, friendly, calm, energetic, or deep This makes it much easier to find a voice for a specific application, for example: • Conversational + warm + middle-aged • Narration + deep + confident • Social media + young + energetic • Educational + calm + friendly We manually selected, reviewed, and tagged the voices so the metadata reflects how the voices actually sound and where they work well. We also redesigned the TTS Playground to make it easier to browse, filter, compare, and test voices with your own text and languages. Learn more:

Soniox

126,350 views • 29 days ago

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.
0:36

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,956 views • 2 years ago

Today we're announcing $280M in Series B funding at a $2B valuation, led by our long-time investor and partner, Menlo Ventures. When we announced our Series A last May, most conversations I had about voice began with someone explaining why they doubted it. I don't have those conversations anymore. People tell me instead how much time they save talking instead of typing and what they want us to build next. In a little over a year, voice has gone from something people were questioning to something they rely on, and it happened faster than we expected. That shift is why we've raised a new round of funding. This funding represents a deeper investment in our products, lab, models, and our team. Alongside the funding, we're announcing a preview of our first proprietary speech model, Canto. Canto is a 2B parameter speech model trained for the places people actually talk: loud rooms, windy streets, a toddler in the background, a second language mixed into the first. Models like this will change how we all interact with devices, and Canto is the first in a rapid line of them. Each one will be larger and more capable than the last, and each should improve Flow's dictation in a way you can feel the day it ships. Larger speech models also open the door to the vision we dreamed up 5 years ago: using your voice as the primary way you interact with devices. That still has to be invented. A voice interface people rely on all day, in every app, doesn't exist anywhere yet. It's time for that to change. To Sahaj Garg, our team, and every person who gave Wispr Flow a chance - this one's for you.
1:00

Today we're announcing $280M in Series B funding at a $2B valuation, led by our long-time investor and partner, Menlo Ventures. When we announced our Series A last May, most conversations I had about voice began with someone explaining why they doubted it. I don't have those conversations anymore. People tell me instead how much time they save talking instead of typing and what they want us to build next. In a little over a year, voice has gone from something people were questioning to something they rely on, and it happened faster than we expected. That shift is why we've raised a new round of funding. This funding represents a deeper investment in our products, lab, models, and our team. Alongside the funding, we're announcing a preview of our first proprietary speech model, Canto. Canto is a 2B parameter speech model trained for the places people actually talk: loud rooms, windy streets, a toddler in the background, a second language mixed into the first. Models like this will change how we all interact with devices, and Canto is the first in a rapid line of them. Each one will be larger and more capable than the last, and each should improve Flow's dictation in a way you can feel the day it ships. Larger speech models also open the door to the vision we dreamed up 5 years ago: using your voice as the primary way you interact with devices. That still has to be invented. A voice interface people rely on all day, in every app, doesn't exist anywhere yet. It's time for that to change. To Sahaj Garg, our team, and every person who gave Wispr Flow a chance - this one's for you.

Tanay Kothari

615,867 views • 1 month ago

🇺🇸 WHEN NEWS BREAKS, REAL AMERICA’S VOICE WILL BE THERE. From the White House to the road with President Trump, RAV is putting cameras on the ground and bringing viewers presidential coverage as it happens. And when other media outlets need access to that footage, RAV is making its pool coverage available to them. This is the next chapter for Real America’s Voice — powered by a team that never stops and the viewers who make it possible. We are Real America’s Voice — and we’re proud of it. 🇺🇸 Jack Posobiec Nick Sortor John Solomon Bannon’s WarRoom David Zere @ Real Americas Voice News - RAV-TV Dr. Gina Grant Stinchfield
1:00

🇺🇸 WHEN NEWS BREAKS, REAL AMERICA’S VOICE WILL BE THERE. From the White House to the road with President Trump, RAV is putting cameras on the ground and bringing viewers presidential coverage as it happens. And when other media outlets need access to that footage, RAV is making its pool coverage available to them. This is the next chapter for Real America’s Voice — powered by a team that never stops and the viewers who make it possible. We are Real America’s Voice — and we’re proud of it. 🇺🇸 Jack Posobiec Nick Sortor John Solomon Bannon’s WarRoom David Zere @ Real Americas Voice News - RAV-TV Dr. Gina Grant Stinchfield

Real America's Voice (RAV)

147,057 views • 16 days ago

How did a tiny, scrappy team build one of the most powerful AI voice models? In a deep dive with Sesame CTO Ankit Kumar and a16z's Anjney Midha, we explore how Sesame is pushing the boundaries of AI conversation, why it open-sourced its speech generation model, and the power of small teams to outdo much larger AI labs on product focus. A part of their secret: a relentless focus on real-time, natural conversations over raw intelligence, and a deep commitment to voice, personality, and user experience. By opening up its speech generation model, Sesame is paving the way for even more breakthroughs in AI-native conversation 👇
1:04

How did a tiny, scrappy team build one of the most powerful AI voice models? In a deep dive with Sesame CTO Ankit Kumar and a16z's Anjney Midha, we explore how Sesame is pushing the boundaries of AI conversation, why it open-sourced its speech generation model, and the power of small teams to outdo much larger AI labs on product focus. A part of their secret: a relentless focus on real-time, natural conversations over raw intelligence, and a deep commitment to voice, personality, and user experience. By opening up its speech generation model, Sesame is paving the way for even more breakthroughs in AI-native conversation 👇

a16z

29,930 views • 1 year ago

Today, we’re introducing new tools and model updates to help you build, deploy, and scale Voice AI applications.🎙️ 🆕 Speech Understanding: Turn transcripts into actionable data with speaker identification, custom formatting & translation 🆕 LLM Gateway: One API for your voice-to-intelligence pipeline, with GPT, Claude, Gemini & more 🆕 Voice AI Guardrails: End-to-end protection for safe, compliant content Model Upgrades: ✅ 99 languages with auto code-switching ✅ 64% fewer speaker counting errors ✅ 57% better accuracy on critical terms with 1,000-word context Real results: 🔥 Calabrio: 80% boost in customer satisfaction, 22% revenue increase 🔥 Siro: "10/10 customers say 'wow, that insight was crisp'" Build & scale Voice AI in minutes! Try it in our Playground or check the docs. 🚀
0:58

Today, we’re introducing new tools and model updates to help you build, deploy, and scale Voice AI applications.🎙️ 🆕 Speech Understanding: Turn transcripts into actionable data with speaker identification, custom formatting & translation 🆕 LLM Gateway: One API for your voice-to-intelligence pipeline, with GPT, Claude, Gemini & more 🆕 Voice AI Guardrails: End-to-end protection for safe, compliant content Model Upgrades: ✅ 99 languages with auto code-switching ✅ 64% fewer speaker counting errors ✅ 57% better accuracy on critical terms with 1,000-word context Real results: 🔥 Calabrio: 80% boost in customer satisfaction, 22% revenue increase 🔥 Siro: "10/10 customers say 'wow, that insight was crisp'" Build & scale Voice AI in minutes! Try it in our Playground or check the docs. 🚀

AssemblyAI

18,157 views • 11 months ago

The best way to take advantage of ChatGPT Voice. I am completely and utterly voice-pilled. Voice interaction with AI is going to change the way humans interact with computers. Here's what you can do: 1. Create a remote connection to an always-on desktop PC through the ChatGPT Desktop app (I have a Linux box in my basement) 2. Customize the ChatGPT iOS widget and add a remote voice connection 3. Tap the widget. It will open a voice chat in ChatGPT Work/Codex, on your desktop, with access to your files and all your apps. It's like an always on Jarvis that has its own computer. Amazing stuff. Try it out!
0:28

The best way to take advantage of ChatGPT Voice. I am completely and utterly voice-pilled. Voice interaction with AI is going to change the way humans interact with computers. Here's what you can do: 1. Create a remote connection to an always-on desktop PC through the ChatGPT Desktop app (I have a Linux box in my basement) 2. Customize the ChatGPT iOS widget and add a remote voice connection 3. Tap the widget. It will open a voice chat in ChatGPT Work/Codex, on your desktop, with access to your files and all your apps. It's like an always on Jarvis that has its own computer. Amazing stuff. Try it out!

Dan McAteer

46,466 views • 1 month ago

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:
2:41

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:

Andrew Ng

87,965 views • 1 year ago