正在加载视频...

视频加载失败

返回首页

What will you build with Vision Agents? Out-of-the-box support for: - Turn detection - Speech-to-text + text-to-speech - Voice activity detection - MCP & function-calling support Open-source. Video-first. Ready to build.

getstream_io's profile picture

Stream

12,373 subscribers

226,723 次观看 • 7 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️
4:29

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️

Prince Canuma

163,831 次观看 • 5 个月前

Flux Multilingual is live. Real-time conversational speech-to-text for voice agents in 10 languages, with monolingual-grade accuracy, turn detection, and code-switching. Deploy once and launch globally. Learn more →
1:13

Flux Multilingual is live. Real-time conversational speech-to-text for voice agents in 10 languages, with monolingual-grade accuracy, turn detection, and code-switching. Deploy once and launch globally. Learn more →

Deepgram

15,544 次观看 • 3 个月前

SwiftUI: You can chain free and open-source STT, TTS, and LLMs to build voice/vision apps with Core AI. Here is a demo I was experimenting with: - Whisper: Speech-to-text - Kokoro-82M: Text-to-speech - Qwen3-VL 2B: Vision I converted Pocket TTS and VibeVoice to Core AI models and tried with Gemma 4. All is working great.
0:36

SwiftUI: You can chain free and open-source STT, TTS, and LLMs to build voice/vision apps with Core AI. Here is a demo I was experimenting with: - Whisper: Speech-to-text - Kokoro-82M: Text-to-speech - Qwen3-VL 2B: Vision I converted Pocket TTS and VibeVoice to Core AI models and tried with Gemma 4. All is working great.

Amos Gyamfi

79,756 次观看 • 1 个月前

Say It — private, local text-to-speech for macOS Open models 100% on-device, speak text from any app with a hotkey, and even clone your own voice. Infinite voices. Free & open-source. Listen to the video 🔊
0:23

Say It — private, local text-to-speech for macOS Open models 100% on-device, speak text from any app with a hotkey, and even clone your own voice. Infinite voices. Free & open-source. Listen to the video 🔊

calle 🟥

34,099 次观看 • 10 天前

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.
4:18

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.

kwindla

282,617 次观看 • 2 年前

Want to add offline voice AI to your ESP32? Start with ESP-Skainet. It is an open-source speech AI framework for building always-on voice assistants that run entirely on ESP32 devices. It combines WakeNet for wake-word detection, MultiNet for offline speech commands, and an Audio Front End with echo cancellation, noise suppression, and voice activity detection. ESP32-S3 is the recommended platform, with the latest models also targeting ESP32-P4.
0:20

Want to add offline voice AI to your ESP32? Start with ESP-Skainet. It is an open-source speech AI framework for building always-on voice assistants that run entirely on ESP32 devices. It combines WakeNet for wake-word detection, MultiNet for offline speech commands, and an Audio Front End with echo cancellation, noise suppression, and voice activity detection. ESP32-S3 is the recommended platform, with the latest models also targeting ESP32-P4.

Praveen Kumar Verma

15,008 次观看 • 25 天前

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:
2:41

Learn to build conversational AI voice agents in "Building AI Voice Agents for Production", created in collaboration with LiveKit and RealAvatar, and taught by dsa (Co-founder & CEO of LiveKit), Shayne (Developer Advocate, LiveKit), and Nedelina Teneva (Head of AI at RealAvatar, an AI Fund portfolio company). Voice agents combine speech and reasoning capabilities to enable real-time conversations. They're already being used to support customer service, to improve accessibility in healthcare, for entertainment applications, and for talk therapy. In this course, you’ll learn to build voice agents that listen, reason, and respond naturally. You’ll follow the architecture used to create the "AI Andrew" Avatar, a collaborative project between and RealAvatar that responds to users in what sounds like my voice. You’ll build a voice agent from scratch and deploy it to the cloud, enabling support for many simultaneous users. What you’ll learn: - Understand the fundamentals of voice agents, including key components like speech-to-text (STT), text-to-speech (TTS), and LLMs, and how latency is introduced at each layer. - Explore voice agent architectures and the trade-offs between modular pipelines and speech-to-speech APIs. - Explore how platforms like LiveKit mitigate latency issues with optimized networking infrastructure and low-latency communication protocols. - Learn how to connect client devices to voice agents using WebRTC—and why it outperforms HTTP and WebSocket for low-latency audio streaming. - Incorporate voice activity detection (VAD), end-of-turn detection, and context management to detect turns, handle interruptions, and manage conversational flow. - Understand the trade-offs between latency, quality, and cost in an example in which you build a voice agent and change its voice. - Equip your agent with metrics to measure latency at each stage of the voice pipeline and learn the key levers you can pull to make your agent faster and more responsive. The voice agents built in this course also incorporate voice technology from , a supporting contributor to the project. By the end of this course, you'll have learned the components of an AI voice agent pipeline, combined them into a system with low-latency communication, and deployed them on cloud infrastructure so it scales to many users. I’m looking forward to seeing what voice agents you build from this course! Please sign up here:

Andrew Ng

87,701 次观看 • 1 年前

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!
1:17

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!

Andi Marafioti

109,602 次观看 • 10 天前

MyRobot Dev Blog 1/14/26 - Quality of Life I added offline text to speech support (Chatterbox) so you can host your own offline text to speech models.
0:45

MyRobot Dev Blog 1/14/26 - Quality of Life I added offline text to speech support (Chatterbox) so you can host your own offline text to speech models.

sgthale | MyRobot Dev

42,817 次观看 • 5 个月前

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️
0:56

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

AI at Meta

592,782 次观看 • 3 年前

Realtime speech-to-text for developers building conversational apps, AI agents, and voice workflows.
0:30

Realtime speech-to-text for developers building conversational apps, AI agents, and voice workflows.

ElevenLabs Developers

6,620,525 次观看 • 3 个月前

Oh my…FlashLabs releases Chroma 1.0, first open-source real-time speech-to-speech model with personalized voice cloning. Native speech-to-speech with <150ms latency & voice cloning from seconds of audio. Finally, an open alternative to OpenAI Realtime.
1:57

Oh my…FlashLabs releases Chroma 1.0, first open-source real-time speech-to-speech model with personalized voice cloning. Native speech-to-speech with <150ms latency & voice cloning from seconds of audio. Finally, an open alternative to OpenAI Realtime.

Alvaro Cintas

46,151 次观看 • 7 个月前

Presenting MetaVoice-1B, a 1.2B parameter base model for TTS (text-to-speech). * Emotional speech in English * Voice cloning with fine-tuning * Zero-shot cloning for American & British voices * Support for long-form synthesis
1:54

Presenting MetaVoice-1B, a 1.2B parameter base model for TTS (text-to-speech). * Emotional speech in English * Voice cloning with fine-tuning * Zero-shot cloning for American & British voices * Support for long-form synthesis

MetaVoice

111,836 次观看 • 2 年前

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️
1:30

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️

AI at Meta

326,883 次观看 • 3 年前

i gave a voice to my jarvis system everything running in real-time using open source models - roboflow RF-DETR for object detection - gemma 4 for scene summarization - kokoro text-to-speech - live in the browser using transformers js prompt: "you are a dystopian science fiction narrator in a cyberpunk world. describe the scene in one sentence"
0:24

i gave a voice to my jarvis system everything running in real-time using open source models - roboflow RF-DETR for object detection - gemma 4 for scene summarization - kokoro text-to-speech - live in the browser using transformers js prompt: "you are a dystopian science fiction narrator in a cyberpunk world. describe the scene in one sentence"

AA

28,753 次观看 • 4 个月前

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.
0:36

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,764 次观看 • 1 年前

ComfyUI MCP server - A project from Pixelle AI Shoutout to Pixelle-MCP - an amazing open-source project that bridges ComfyUI with LLMs through the MCP protocol. Zero-code, talk to your LLM to iterate ComfyUI workflow! Highlights: - Full TISV (Text, Image, Sound/Speech, Video) support - Flexible deployment options - Easy setup with Docker or one-click scripts - Custom workflow tools in just a few steps 🧵👇
1:00

ComfyUI MCP server - A project from Pixelle AI Shoutout to Pixelle-MCP - an amazing open-source project that bridges ComfyUI with LLMs through the MCP protocol. Zero-code, talk to your LLM to iterate ComfyUI workflow! Highlights: - Full TISV (Text, Image, Sound/Speech, Video) support - Flexible deployment options - Easy setup with Docker or one-click scripts - Custom workflow tools in just a few steps 🧵👇

ComfyUI

27,949 次观看 • 1 年前

✨ bolt.new + BuildShip = Voice-to-Voice Translation App 🎙️ 🚀 Build your own AI-powered translator: - Convert speech → text (OpenAI Whisper) - Translate into any language - Generate lifelike speech (TTS) - Upload & retrieve translated audio files
0:24

✨ bolt.new + BuildShip = Voice-to-Voice Translation App 🎙️ 🚀 Build your own AI-powered translator: - Convert speech → text (OpenAI Whisper) - Translate into any language - Generate lifelike speech (TTS) - Upload & retrieve translated audio files

BuildShip

12,742 次观看 • 1 年前

I made my Twitch Chat voice Half-Life 2 NPCs. They read out lines with text-to-speech in a recording studio, then I import their text-to-speech into the game.
0:22

I made my Twitch Chat voice Half-Life 2 NPCs. They read out lines with text-to-speech in a recording studio, then I import their text-to-speech into the game.

shindigs

19,218 次观看 • 1 个月前