Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

I built a simple voice assistant in 70 lines of Python code. It uses: • LiveKit - The voice agent • AssemblyAI - To turn your voice into text • OpenAI - The brain of the agent, and to turn text into audio There's something really cool about this:

34,718 Aufrufe • vor 1 Jahr •via X (Twitter)

22 Kommentare

Profilbild von Santiago
Santiagovor 1 Jahr

I'm using @AssemblyAI's turn detection, and it's really good! Turn detection tells the assistant when the person stops talking, and it's time for the other person to talk. Before, we used pauses and silence to detect turns, but that's not a great approach (imagine somebody making a pause while dictating a credit card). The new turn detection version is much more robust. Run this code and talk to the assistant. This feels way more natural than ever before. Here is the repository: You'll find the instructions to run the code in the README file. If you prefer to watch the video on YouTube, here is the link:

Profilbild von Alex Veremeyenko
Alex Veremeyenkovor 1 Jahr

@livekit @AssemblyAI @OpenAI that’s impressive for just 70 lines, leveraging those tools really enhances the experience. clean code can go a long way in simplifying voice interactions.

Profilbild von kavin karthick
kavin karthickvor 1 Jahr

@livekit @AssemblyAI @OpenAI livekit is so good ...

Profilbild von Saïd Aitmbarek
Saïd Aitmbarekvor 1 Jahr

@livekit @AssemblyAI @OpenAI so cool, gotta try livekit!

Profilbild von Nakesh Singh
Nakesh Singhvor 1 Jahr

@livekit @AssemblyAI @OpenAI Great content. Do we need to purchase subscription for @livekit, AssemblyAI, openAI ??

Profilbild von Robert Youssef
Robert Youssefvor 1 Jahr

@livekit @AssemblyAI @OpenAI nice work, love how you leveraged those tools in so few lines. that voice agent setup looks clean

Profilbild von ___raf stahelin___
___raf stahelin___vor 1 Jahr

@livekit @AssemblyAI @OpenAI Can it be used with Claude code?

Profilbild von Nayo
Nayovor 1 Jahr

@livekit @AssemblyAI @OpenAI Simplicity fucks

Profilbild von AI Expert
AI Expertvor 1 Jahr

@livekit @AssemblyAI @OpenAI Great 👍🏽👍🏽

Profilbild von Akanksha Soni
Akanksha Sonivor 1 Jahr

@livekit @AssemblyAI @OpenAI Hii guys I have started a 100 Days DSA Challange .. So is there anyone who would like to join along with me in this journey .. Lets leran together 🔥

Profilbild von Pyxis
Pyxisvor 1 Jahr

@livekit @AssemblyAI @OpenAI I knew the answer immediately. What’s the point?

Profilbild von MangoMagic™️
MangoMagic™️vor 1 Jahr

@livekit @AssemblyAI @OpenAI Impressive. Python just got a whole lot more chatty.

Profilbild von sc
scvor 1 Jahr

@livekit @AssemblyAI @OpenAI To use connected online services is easy as of today. Build a local one no data streams leave your pc. We could use as example STT Vosk, LM Studio local model and TTS Mimic3… when you dive into the details it gets very interesting and we understand process power is limit.

Profilbild von TensorFlow
TensorFlowvor 1 Jahr

@livekit @AssemblyAI @OpenAI Would be more impressive without paying for the usage of each line of code and getting the same quality

Profilbild von Hunter
Huntervor 1 Jahr

@livekit @AssemblyAI @OpenAI Can you do one with real time api? Setting up web sockets is a pain.

Profilbild von Sri Cherukuri
Sri Cherukurivor 1 Jahr

@livekit @AssemblyAI @OpenAI Cool. I work on similar voice agents but for telephony calls. I use the following: Amazon Connect Deepgram ElevenLabs

Profilbild von Musu Priyam MP
Musu Priyam MPvor 1 Jahr

@livekit @AssemblyAI @OpenAI One question. How do you test it when at scale? Even I thought there is no end-to-end testing platform, but I've been using this simulate app for the past few weeks by future agi. As an AI developer, I can tell this is immensely helpful.

Profilbild von JoshSato
JoshSatovor 1 Jahr

@livekit @AssemblyAI @OpenAI 70 lines is wild. the brain part is always the coolest when it just clicks like that.

Profilbild von Himanshu Kumar
Himanshu Kumarvor 1 Jahr

@livekit @AssemblyAI @OpenAI Impressive, esp for 70 lines.. Consider fine-tuning the speech-to-text model, tho.. Unlocks smoother, more natural interactions often overlooked..

Profilbild von Wade Watts
Wade Wattsvor 1 Jahr

@livekit @AssemblyAI @OpenAI Your AI needs more character. At least have a pirate accent.

Profilbild von AI Alchemy
AI Alchemyvor 1 Jahr

@livekit @AssemblyAI @OpenAI Helpful!

Profilbild von Ian Brown
Ian Brownvor 1 Jahr

@livekit @AssemblyAI @OpenAI This is the end of service centre/call centre jobs...

Ähnliche Videos

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:

Andrew Ng

87,965 Aufrufe • vor 1 Jahr