正在加载视频...

视频加载失败

I built a simple voice assistant in 70 lines of Python code. It uses: • LiveKit - The voice agent • AssemblyAI - To turn your voice into text • OpenAI - The brain of the agent, and to turn text into audio There's something really cool about this:

34,718 次观看 • 1 年前 •via X (Twitter)

22 条评论

Santiago 的头像
Santiago1 年前

I'm using @AssemblyAI's turn detection, and it's really good! Turn detection tells the assistant when the person stops talking, and it's time for the other person to talk. Before, we used pauses and silence to detect turns, but that's not a great approach (imagine somebody making a pause while dictating a credit card). The new turn detection version is much more robust. Run this code and talk to the assistant. This feels way more natural than ever before. Here is the repository: You'll find the instructions to run the code in the README file. If you prefer to watch the video on YouTube, here is the link:

Alex Veremeyenko 的头像
Alex Veremeyenko1 年前

@livekit @AssemblyAI @OpenAI that’s impressive for just 70 lines, leveraging those tools really enhances the experience. clean code can go a long way in simplifying voice interactions.

kavin karthick 的头像
kavin karthick1 年前

@livekit @AssemblyAI @OpenAI livekit is so good ...

Saïd Aitmbarek 的头像
Saïd Aitmbarek1 年前

@livekit @AssemblyAI @OpenAI so cool, gotta try livekit!

Nakesh Singh 的头像
Nakesh Singh1 年前

@livekit @AssemblyAI @OpenAI Great content. Do we need to purchase subscription for @livekit, AssemblyAI, openAI ??

Robert Youssef 的头像
Robert Youssef1 年前

@livekit @AssemblyAI @OpenAI nice work, love how you leveraged those tools in so few lines. that voice agent setup looks clean

___raf stahelin___ 的头像
___raf stahelin___1 年前

@livekit @AssemblyAI @OpenAI Can it be used with Claude code?

Nayo 的头像
Nayo1 年前

@livekit @AssemblyAI @OpenAI Simplicity fucks

AI Expert 的头像
AI Expert1 年前

@livekit @AssemblyAI @OpenAI Great 👍🏽👍🏽

Akanksha Soni 的头像
Akanksha Soni1 年前

@livekit @AssemblyAI @OpenAI Hii guys I have started a 100 Days DSA Challange .. So is there anyone who would like to join along with me in this journey .. Lets leran together 🔥

Pyxis 的头像
Pyxis1 年前

@livekit @AssemblyAI @OpenAI I knew the answer immediately. What’s the point?

MangoMagic™️ 的头像
MangoMagic™️1 年前

@livekit @AssemblyAI @OpenAI Impressive. Python just got a whole lot more chatty.

sc 的头像
sc1 年前

@livekit @AssemblyAI @OpenAI To use connected online services is easy as of today. Build a local one no data streams leave your pc. We could use as example STT Vosk, LM Studio local model and TTS Mimic3… when you dive into the details it gets very interesting and we understand process power is limit.

TensorFlow 的头像
TensorFlow1 年前

@livekit @AssemblyAI @OpenAI Would be more impressive without paying for the usage of each line of code and getting the same quality

Hunter 的头像
Hunter1 年前

@livekit @AssemblyAI @OpenAI Can you do one with real time api? Setting up web sockets is a pain.

Sri Cherukuri 的头像
Sri Cherukuri1 年前

@livekit @AssemblyAI @OpenAI Cool. I work on similar voice agents but for telephony calls. I use the following: Amazon Connect Deepgram ElevenLabs

Musu Priyam MP 的头像
Musu Priyam MP1 年前

@livekit @AssemblyAI @OpenAI One question. How do you test it when at scale? Even I thought there is no end-to-end testing platform, but I've been using this simulate app for the past few weeks by future agi. As an AI developer, I can tell this is immensely helpful.

JoshSato 的头像
JoshSato1 年前

@livekit @AssemblyAI @OpenAI 70 lines is wild. the brain part is always the coolest when it just clicks like that.

Himanshu Kumar 的头像
Himanshu Kumar1 年前

@livekit @AssemblyAI @OpenAI Impressive, esp for 70 lines.. Consider fine-tuning the speech-to-text model, tho.. Unlocks smoother, more natural interactions often overlooked..

Wade Watts 的头像
Wade Watts1 年前

@livekit @AssemblyAI @OpenAI Your AI needs more character. At least have a pirate accent.

AI Alchemy 的头像
AI Alchemy1 年前

@livekit @AssemblyAI @OpenAI Helpful!

Ian Brown 的头像
Ian Brown1 年前

@livekit @AssemblyAI @OpenAI This is the end of service centre/call centre jobs...

相关视频

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:

Andrew Ng

87,965 次观看 • 1 年前