Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

This is the fastest text-to-speech and speech-to-text API out there. It's under 250 ms to first-byte latency. That's what we need to deploy conversational AI applications everywhere! (I can't wait for the death of automated voice machines.) Look at the attached video. For the first time, I could have... a conversation with an AI agent in real time without awkward pauses. This technology is Aura from Deepgram. It's the fastest text-to-speech option I've seen. Here is a GitHub repository with the demo in the video: You can use it to build a conversational AI application using natural voice. You'll need a Deepgram API Key and an OpenAI API Key. Thanks to Deepgram for giving me access to Aura and collaborating with me on this post. If you want to learn more about Aura, here is the announcement post:show more

Santiago

451,853 subscribers

185,560 views • 2 years ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

jordan rothstein ☎️2 years ago

1. Use Groq for even faster inference 2. Connect with Twilio 3. IVR is gone

Vaibhav2 years ago

Absolutely groundbreaking! Aura's sub-250 ms latency is a game-changer for conversational AI, bringing us closer to seamless, real-time dialogues with AI agents. Can't wait to dive into the demo and see this in action. The future of AI communication is here! #Deepgram #AIRevolution

Vaibhav2 years ago

Real-time voice agents need to be quick and natural. Aura's sub-250ms response times for dialogue sequences ensure conversations flow smoothly, without the awkward pauses of yesteryear's tech.

Vaibhav2 years ago

1/5 Deepgram's Aura is shaking up the text-to-speech scene, boasting the lowest latency for real-time voice AI. With human-like voices and lightning-fast responses, it's set to redefine conversational AI.

Osmosis2 years ago

We can't recommend Deepgram enough. This team is awesome!

OP. Collins2 years ago

Check out Bland, just as good or even better

RameshR2 years ago

I will kill GUI

bog123452 years ago

Think of the application for translation of text and back to speech in a different language

synthian wall facer2 years ago

banger

Vaibhav2 years ago

With Aura, Deepgram rounds out its Voice AI Platform, offering a one-stop-shop for devs to craft immersive voice AI experiences. From transcribing speech to generating lifelike voice responses, it's all here.

Related Videos

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

Melvin Vivas

27,722 views • 1 year ago

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:

Andrew Ng

87,484 views • 1 year ago

Releasing the simplest way to build low latency phone agent on Nexus - live in 2 clicks: Can transfer call, send text messages, access all your knowledge and use any of the +1500 plugins available on nexus. - 📞 Able to buy any number in the world thanks to twilio - 🗣️ State of the art TTS with choice of voice from , and OpenAI - 💬 Low latency STT thanks to Deepgram 🎼 Orchestration thanks to a partnership with Millis AI Here is an example of a personal assistant able to screen calls and access my agenda

Releasing the simplest way to build low latency phone agent on Nexus - live in 2 clicks: Can transfer call, send text messages, access all your knowledge and use any of the +1500 plugins available on nexus. - 📞 Able to buy any number in the world thanks to twilio - 🗣️ State of the art TTS with choice of voice from , and OpenAI - 💬 Low latency STT thanks to Deepgram 🎼 Orchestration thanks to a partnership with Millis AI Here is an example of a personal assistant able to screen calls and access my agenda

Assem

13,015 views • 1 year ago

108 workflow templates you can use to build AI applications without writing any code. You can use these templates with n8n. I recorded the attached video to show you how it works. n8n is the workhorse behind an open-source, self-hosted AI starter kit you can install on your computer. They are sponsoring this post. Here is the link to the starter kit repository: And here is the spreadsheet with the 108 templates: Whatever idea you have, search for something similar in the list of templates, and you'll save a ton of time. Lately, I've talked to many non-coders who want to start using AI more seriously to build things. n8n is perfect for that.

108 workflow templates you can use to build AI applications without writing any code. You can use these templates with n8n. I recorded the attached video to show you how it works. n8n is the workhorse behind an open-source, self-hosted AI starter kit you can install on your computer. They are sponsoring this post. Here is the link to the starter kit repository: And here is the spreadsheet with the 108 templates: Whatever idea you have, search for something similar in the list of templates, and you'll save a ton of time. Lately, I've talked to many non-coders who want to start using AI more seriously to build things. n8n is perfect for that.

Santiago

78,133 views • 1 year ago

Postman is going all-in on AI! Their AI Agent Builder is live (and it's pretty powerful!) Postman's API Network has 100,000+ APIs and they just released a suite of tools for building AI Agents that can automatically discover and connect to any of these APIs. Think about it for a second: You can now build an AI agent that searches the network, discovers an API, processes its documentation, connects to it, and evaluates whether everything worked at the end of the process. This is a huge deal. Every person I know uses the Postman client to work with APIs. Now, they will have access to the AI Agent Builder, a complete suite for designing, building, testing, and deploying agents. I've built a few simple agents myself, and it's tough. The ability to discover and use an API automatically is really exciting (and a huge time saver if it works as promised!) In collaboration with the Postman team, who is sponsoring this post, I recorded a video with my thoughts about the Postman AI Agent Builder.

Postman is going all-in on AI! Their AI Agent Builder is live (and it's pretty powerful!) Postman's API Network has 100,000+ APIs and they just released a suite of tools for building AI Agents that can automatically discover and connect to any of these APIs. Think about it for a second: You can now build an AI agent that searches the network, discovers an API, processes its documentation, connects to it, and evaluates whether everything worked at the end of the process. This is a huge deal. Every person I know uses the Postman client to work with APIs. Now, they will have access to the AI Agent Builder, a complete suite for designing, building, testing, and deploying agents. I've built a few simple agents myself, and it's tough. The ability to discover and use an API automatically is really exciting (and a huge time saver if it works as promised!) In collaboration with the Postman team, who is sponsoring this post, I recorded a video with my thoughts about the Postman AI Agent Builder.

Santiago

230,571 views • 1 year ago

Building with AI gets easier every day. Here is an open-source library that makes integrating AI into an application extremely easy: Star the repository! This library alone can make React the best front-end framework out there! There are a bunch of cool things I like about CopilotKit. Here are 3 of them: 1. It allows you to take any -powered agent and bring it into your application. (This is a brand-new feature!) 2. You can build an AI-powered chatbot in your application. The chatbot will have access to your context and can act on the application. 3. You can build a RAG workflow to process and answer questions from a real-time knowledge base. I recorded a video to show you how simple it is to make some of this happen. A few lines of code, and you are in business. Here is a link to the sample application: CopilotKit is open-source. You can self-host it. You can use it with any LLM. Thanks to the team for showing me their tool and collaborating with me on this post!

Santiago

108,824 views • 2 years ago

Build an open-source AI video studio with this Next.js template using Veo 3 and Imagen 4 in the Gemini API. Create text-to-video, image-to-video, and edit videos in the browser for a specific time range.

Build an open-source AI video studio with this Next.js template using Veo 3 and Imagen 4 in the Gemini API. Create text-to-video, image-to-video, and edit videos in the browser for a specific time range.

Google AI Developers

41,423 views • 10 months ago

Imagine an AI application that can type anywhere you can and use the full context of what's on your screen. This is the application we all deserve (at least if you have macOS.) Check out Omnipilot. It's an app that works with every other macOS application and uses Claude Sonet 3.5 in the background—it also supports Gemini and GPT-4o. Here is the idea: You can use the tool to ask questions about anything on your screen. Or you can use it to autocomplete the text you are typing. You don't need to copy and paste anymore or waste your time providing context to a model. It sees what you see. It works right where you are. That's pretty cool. Here are a couple of cool examples: • Use it to reply to an email • Use it in the terminal to autocomplete a command • Use it to finish a document • Use to send a message on Slack AI at the system level is bonkers. You can read a ton more on their Product Hunt launch page: Thanks to the Omnipilot team for collaborating with me on this post!

Imagine an AI application that can type anywhere you can and use the full context of what's on your screen. This is the application we all deserve (at least if you have macOS.) Check out Omnipilot. It's an app that works with every other macOS application and uses Claude Sonet 3.5 in the background—it also supports Gemini and GPT-4o. Here is the idea: You can use the tool to ask questions about anything on your screen. Or you can use it to autocomplete the text you are typing. You don't need to copy and paste anymore or waste your time providing context to a model. It sees what you see. It works right where you are. That's pretty cool. Here are a couple of cool examples: • Use it to reply to an email • Use it in the terminal to autocomplete a command • Use it to finish a document • Use to send a message on Slack AI at the system level is bonkers. You can read a ton more on their Product Hunt launch page: Thanks to the Omnipilot team for collaborating with me on this post!

Santiago

72,306 views • 1 year ago

The call center business is in real trouble. Deepgram just released an API for real conversations. It's a serious upgrade for anyone building voice-first applications. This is no longer about theatrical narration, but about real conversations. Here are the details: 1/5

The call center business is in real trouble. Deepgram just released an API for real conversations. It's a serious upgrade for anyone building voice-first applications. This is no longer about theatrical narration, but about real conversations. Here are the details: 1/5

Santiago

48,285 views • 1 year ago

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.

kwindla

282,539 views • 2 years ago

Voice agents are awkward, and everyone notices: You ask a question. The agent thinks. You wait. And wait... Nobody wants this. I'd rather talk to a person. If your model's response time is over 300ms, you won't make it. Unfortunately, most text-to-speech models can't get anywhere close to that. I want you to take a look at the latest model released by Inworld AI: TTS-1.5. I built a simple voice agent using the model so you can see it in action and test it on your computer. You'll find the repository link below. The latency numbers of this model are wild: • Max model → under 250ms • Mini model → under 130ms That's 4x faster than prior generations and faster than human response times!

Voice agents are awkward, and everyone notices: You ask a question. The agent thinks. You wait. And wait... Nobody wants this. I'd rather talk to a person. If your model's response time is over 300ms, you won't make it. Unfortunately, most text-to-speech models can't get anywhere close to that. I want you to take a look at the latest model released by Inworld AI: TTS-1.5. I built a simple voice agent using the model so you can see it in action and test it on your computer. You'll find the repository link below. The latency numbers of this model are wild: • Max model → under 250ms • Mini model → under 130ms That's 4x faster than prior generations and faster than human response times!

Santiago

69,418 views • 5 months ago

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Vaibhav Sisinty

24,526,445 views • 2 months ago

Here is the fastest way to turn text into a voice. Less than 100ms to start streaming. This is an absolute must for anyone who wants to build a voice agent that doesn't sound robotic (latency kills the perception of reality). Here is how easy this is:

Here is the fastest way to turn text into a voice. Less than 100ms to start streaming. This is an absolute must for anyone who wants to build a voice agent that doesn't sound robotic (latency kills the perception of reality). Here is how easy this is:

Santiago

38,164 views • 2 months ago

Forget about HTML, JavaScript, and CSS. You can build AI and data applications using Python alone! Python is the best programming language out there. You can now use it across the stack. Take a look at Taipy, an open-source Python library to build end-to-end production applications. Star their repository: I recorded a quick video showing how to build a simple chat interface to talk to OpenAI GPT-3.5 using Taipy alone. Something important to keep in mind: Taipy's goal is not to replace web developers but to provide an alternative to those who need to build applications without web experience. If you are a data scientist or someone dealing with data, Taipy will simplify your life considerably. Thanks to the team behind Taipy for collaborating with me on this post. Here is a link to the full demo. Download this code and run it. It takes 30 seconds:

Forget about HTML, JavaScript, and CSS. You can build AI and data applications using Python alone! Python is the best programming language out there. You can now use it across the stack. Take a look at Taipy, an open-source Python library to build end-to-end production applications. Star their repository: I recorded a quick video showing how to build a simple chat interface to talk to OpenAI GPT-3.5 using Taipy alone. Something important to keep in mind: Taipy's goal is not to replace web developers but to provide an alternative to those who need to build applications without web experience. If you are a data scientist or someone dealing with data, Taipy will simplify your life considerably. Thanks to the team behind Taipy for collaborating with me on this post. Here is a link to the full demo. Download this code and run it. It takes 30 seconds:

Santiago

233,878 views • 2 years ago

I can search a database with 32 million records in less than 50 milliseconds. This speed is insane! There's no reason your search is slow. There's no reason you aren't using this library when building applications with a lot of data. Take a look at typesense |. It's an open-source search engine. It's crazy fast and easy to use. They are sponsoring this post. Here is their GitHub repository: You can use the library in Python, JavaScript, PHP, Ruby, Swift, and pretty much any other programming language out there. Here are some of the things you can do with Typesense: • Vector search over embeddings • Conversational search for RAG applications • Image similarity search • Voice search And it does all of this extremely fast. A few interesting notes about the library: • Optimized for developer happiness • Works out of the box for most use cases • Very intuitive API • Easy to deploy and scale Probably the best way to compare them is against ElasticSearch. The latter is much more complex to configure and run with a steep learning curve. Check their GitHub repository. They have examples, tutorials, and everything you need.

I can search a database with 32 million records in less than 50 milliseconds. This speed is insane! There's no reason your search is slow. There's no reason you aren't using this library when building applications with a lot of data. Take a look at typesense |. It's an open-source search engine. It's crazy fast and easy to use. They are sponsoring this post. Here is their GitHub repository: You can use the library in Python, JavaScript, PHP, Ruby, Swift, and pretty much any other programming language out there. Here are some of the things you can do with Typesense: • Vector search over embeddings • Conversational search for RAG applications • Image similarity search • Voice search And it does all of this extremely fast. A few interesting notes about the library: • Optimized for developer happiness • Works out of the box for most use cases • Very intuitive API • Easy to deploy and scale Probably the best way to compare them is against ElasticSearch. The latter is much more complex to configure and run with a steep learning curve. Check their GitHub repository. They have examples, tutorials, and everything you need.

Santiago

160,154 views • 1 year ago

Here is how simple it is to build a UI front-end for an AI agent (assuming you are using . The video here is 90 seconds and will show you everything you need to connect an agent to a web app: 1. Set up CopilotKit🪁 in your app 1. Copy the agent's localhost URL from LangGraph Studio 2. Paste that URL into a CopilotKit🪁 remote endpoint That's all you need to get a front-end to interact with the agent! The secret sauce here is CoAgents, an open-source library. You can use CoAgents to integrate any agent with your web application. Here is what you get: • Human-in-the-loop to steer and correct the agent • Stream intermediate agent state • Real-time state sharing between the agent and the application • Agentic generative UI to build trust that the agent is on the right path Watch the attached demo so you see it in action.

Here is how simple it is to build a UI front-end for an AI agent (assuming you are using . The video here is 90 seconds and will show you everything you need to connect an agent to a web app: 1. Set up CopilotKit🪁 in your app 1. Copy the agent's localhost URL from LangGraph Studio 2. Paste that URL into a CopilotKit🪁 remote endpoint That's all you need to get a front-end to interact with the agent! The secret sauce here is CoAgents, an open-source library. You can use CoAgents to integrate any agent with your web application. Here is what you get: • Human-in-the-loop to steer and correct the agent • Stream intermediate agent state • Real-time state sharing between the agent and the application • Agentic generative UI to build trust that the agent is on the right path Watch the attached demo so you see it in action.

Santiago

46,109 views • 1 year ago

Introducing Atlas 1. Willow's new frontier speech-to-text model. It outperforms ElevenLabs, Deepgram, OpenAI, and more by a wide margin. Built on the first scalable, human-powered transcription infrastructure ever built for real-time dictation.

Introducing Atlas 1. Willow's new frontier speech-to-text model. It outperforms ElevenLabs, Deepgram, OpenAI, and more by a wide margin. Built on the first scalable, human-powered transcription infrastructure ever built for real-time dictation.

Willow

835,043 views • 3 months ago