Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

What will you build with Vision Agents? Out-of-the-box support for: - Turn detection - Speech-to-text + text-to-speech - Voice activity detection - MCP & function-calling support Open-source. Video-first. Ready to build.

getstream_io's profile picture

Stream

12,373 subscribers

226,723 Aufrufe • vor 6 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️
4:29

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️

Prince Canuma

160,927 Aufrufe • vor 5 Monaten

Flux Multilingual is live. Real-time conversational speech-to-text for voice agents in 10 languages, with monolingual-grade accuracy, turn detection, and code-switching. Deploy once and launch globally. Learn more →
1:13

Flux Multilingual is live. Real-time conversational speech-to-text for voice agents in 10 languages, with monolingual-grade accuracy, turn detection, and code-switching. Deploy once and launch globally. Learn more →

Deepgram

15,544 Aufrufe • vor 2 Monaten

SwiftUI: You can chain free and open-source STT, TTS, and LLMs to build voice/vision apps with Core AI. Here is a demo I was experimenting with: - Whisper: Speech-to-text - Kokoro-82M: Text-to-speech - Qwen3-VL 2B: Vision I converted Pocket TTS and VibeVoice to Core AI models and tried with Gemma 4. All is working great.
0:36

SwiftUI: You can chain free and open-source STT, TTS, and LLMs to build voice/vision apps with Core AI. Here is a demo I was experimenting with: - Whisper: Speech-to-text - Kokoro-82M: Text-to-speech - Qwen3-VL 2B: Vision I converted Pocket TTS and VibeVoice to Core AI models and tried with Gemma 4. All is working great.

Amos Gyamfi

79,756 Aufrufe • vor 25 Tagen

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.
4:18

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.

kwindla

282,617 Aufrufe • vor 2 Jahren

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:
2:41

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:

Andrew Ng

87,484 Aufrufe • vor 1 Jahr

MyRobot Dev Blog 1/14/26 - Quality of Life I added offline text to speech support (Chatterbox) so you can host your own offline text to speech models.
0:45

MyRobot Dev Blog 1/14/26 - Quality of Life I added offline text to speech support (Chatterbox) so you can host your own offline text to speech models.

sgthale | MyRobot Dev

42,817 Aufrufe • vor 4 Monaten

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️
0:56

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

AI at Meta

592,762 Aufrufe • vor 2 Jahren

Oh my…FlashLabs releases Chroma 1.0, first open-source real-time speech-to-speech model with personalized voice cloning. Native speech-to-speech with <150ms latency & voice cloning from seconds of audio. Finally, an open alternative to OpenAI Realtime.
1:57

Oh my…FlashLabs releases Chroma 1.0, first open-source real-time speech-to-speech model with personalized voice cloning. Native speech-to-speech with <150ms latency & voice cloning from seconds of audio. Finally, an open alternative to OpenAI Realtime.

Alvaro Cintas

46,151 Aufrufe • vor 6 Monaten

Presenting MetaVoice-1B, a 1.2B parameter base model for TTS (text-to-speech). * Emotional speech in English * Voice cloning with fine-tuning * Zero-shot cloning for American & British voices * Support for long-form synthesis
1:54

Presenting MetaVoice-1B, a 1.2B parameter base model for TTS (text-to-speech). * Emotional speech in English * Voice cloning with fine-tuning * Zero-shot cloning for American & British voices * Support for long-form synthesis

MetaVoice

111,805 Aufrufe • vor 2 Jahren

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️
1:30

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️

AI at Meta

326,844 Aufrufe • vor 3 Jahren

i gave a voice to my jarvis system everything running in real-time using open source models - roboflow RF-DETR for object detection - gemma 4 for scene summarization - kokoro text-to-speech - live in the browser using transformers js prompt: "you are a dystopian science fiction narrator in a cyberpunk world. describe the scene in one sentence"
0:24

i gave a voice to my jarvis system everything running in real-time using open source models - roboflow RF-DETR for object detection - gemma 4 for scene summarization - kokoro text-to-speech - live in the browser using transformers js prompt: "you are a dystopian science fiction narrator in a cyberpunk world. describe the scene in one sentence"

AA

28,731 Aufrufe • vor 3 Monaten

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.
0:36

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,739 Aufrufe • vor 1 Jahr

ComfyUI MCP server - A project from Pixelle AI Shoutout to Pixelle-MCP - an amazing open-source project that bridges ComfyUI with LLMs through the MCP protocol. Zero-code, talk to your LLM to iterate ComfyUI workflow! Highlights: - Full TISV (Text, Image, Sound/Speech, Video) support - Flexible deployment options - Easy setup with Docker or one-click scripts - Custom workflow tools in just a few steps 🧵👇
1:00

ComfyUI MCP server - A project from Pixelle AI Shoutout to Pixelle-MCP - an amazing open-source project that bridges ComfyUI with LLMs through the MCP protocol. Zero-code, talk to your LLM to iterate ComfyUI workflow! Highlights: - Full TISV (Text, Image, Sound/Speech, Video) support - Flexible deployment options - Easy setup with Docker or one-click scripts - Custom workflow tools in just a few steps 🧵👇

ComfyUI

27,949 Aufrufe • vor 11 Monaten

✨ bolt.new + BuildShip = Voice-to-Voice Translation App 🎙️ 🚀 Build your own AI-powered translator: - Convert speech → text (OpenAI Whisper) - Translate into any language - Generate lifelike speech (TTS) - Upload & retrieve translated audio files
0:24

✨ bolt.new + BuildShip = Voice-to-Voice Translation App 🎙️ 🚀 Build your own AI-powered translator: - Convert speech → text (OpenAI Whisper) - Translate into any language - Generate lifelike speech (TTS) - Upload & retrieve translated audio files

BuildShip

12,742 Aufrufe • vor 1 Jahr

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?
3:29

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

Melvin Vivas

27,722 Aufrufe • vor 1 Jahr

This is the fastest text-to-speech and speech-to-text API out there. It's under 250 ms to first-byte latency. That's what we need to deploy conversational AI applications everywhere! (I can't wait for the death of automated voice machines.) Look at the attached video. For the first time, I could have a conversation with an AI agent in real time without awkward pauses. This technology is Aura from Deepgram. It's the fastest text-to-speech option I've seen. Here is a GitHub repository with the demo in the video: You can use it to build a conversational AI application using natural voice. You'll need a Deepgram API Key and an OpenAI API Key. Thanks to Deepgram for giving me access to Aura and collaborating with me on this post. If you want to learn more about Aura, here is the announcement post:
3:38

This is the fastest text-to-speech and speech-to-text API out there. It's under 250 ms to first-byte latency. That's what we need to deploy conversational AI applications everywhere! (I can't wait for the death of automated voice machines.) Look at the attached video. For the first time, I could have a conversation with an AI agent in real time without awkward pauses. This technology is Aura from Deepgram. It's the fastest text-to-speech option I've seen. Here is a GitHub repository with the demo in the video: You can use it to build a conversational AI application using natural voice. You'll need a Deepgram API Key and an OpenAI API Key. Thanks to Deepgram for giving me access to Aura and collaborating with me on this post. If you want to learn more about Aura, here is the announcement post:

Santiago

185,560 Aufrufe • vor 2 Jahren

🌈' The test of the Intro' I would like to give MC voice-over, what do you think? (Now is text-to-speech for the testing)
3:58

🌈' The test of the Intro' I would like to give MC voice-over, what do you think? (Now is text-to-speech for the testing)

Mr.Ryu

22,813 Aufrufe • vor 1 Jahr

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.
2:47

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

Karan Goel

7,010,735 Aufrufe • vor 1 Monat

One of the most impressive AI demo I've seen. This is the future of customer service. Agents that can understand text, speech, images and even live video. Soon to be all open-source.
4:38

One of the most impressive AI demo I've seen. This is the future of customer service. Agents that can understand text, speech, images and even live video. Soon to be all open-source.

Lior Alexander

270,967 Aufrufe • vor 1 Jahr