Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

🎙️Designing a speech-to-speech assistant Build a speech-to-speech assistant with web search access in 150 lines of code. - Voxtral Transcribe 2 for STT + diarization - Mistral Small 4 for agentic reasoning & efficiency - Voxtral TTS for realistic speech synthesis

MistralDevs's profile picture

Mistral for Developers

12,535 subscribers

18,311 görüntüleme • 7 ay önce •via X (Twitter)

EğitimHaberler & PolitikaBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

11 Yorum

Mistral for Developers profil fotoğrafı
Mistral for Developers7 ay önce

⚙️From STT to our Agent and TTS, pick the best model for your use case and build your own real-time, speech-capable system.

Mistral for Developers profil fotoğrafı
Mistral for Developers7 ay önce

👐Learn more about our audio models - and how to build such pipeline - in the blog post below.

miguel rodriguez profil fotoğrafı
miguel rodriguez7 ay önce

Had a great mistral event today where they showed plenty of features and some interesting use cases. Will proceed with the local integration at Robot Ross: reacts to voice commands. A French accent in a robot just sounds cool.

LightShift Studio profil fotoğrafı
LightShift Studio7 ay önce

Why are you still acting like an old-school phone operator? 🤖 Check out our quick, easy-to-build AI assistant template!

東雲たく profil fotoğrafı
東雲たく7 ay önce

Is it possible for us to run Mistral Small4 and Voxtral TTS on WebGPU?

Raphaël Guintoli profil fotoğrafı
Raphaël Guintoli7 ay önce

Alé mistral

Zer0Z profil fotoğrafı
Zer0Z7 ay önce

and if i want the agent to have my own voice, how do i clone it without online api?

Hyper.AI profil fotoğrafı
Hyper.AI7 ay önce

Building a full S2S assistant in just 150 lines is a massive achievement for dev productivity! 🎙️ The combination of Voxtral for high-fidelity speech and Mistral Small 4 for reasoning makes for an incredibly efficient pipeline.

Victor Peghaire profil fotoğrafı
Victor Peghaire7 ay önce

😍

Carolina profil fotoğrafı
Carolina7 ay önce

Nice stack for a real time S2S pipeline, a solid path to scalable voices and fast web results

cCross profil fotoğrafı
cCross7 ay önce

Nice mix, curious how web search latency holds up

Benzer Videolar

🎙️Do you know you now have all the building blocks for full speech-to-speech? - Voxtral Realtime: High-quality, real-time speech-to-text. - Mistral Small 4: Fast, efficient, general-purpose agentic model. - Voxtral TTS: Realistic customizable text-to-speech with streaming output.
0:50

🎙️Do you know you now have all the building blocks for full speech-to-speech? - Voxtral Realtime: High-quality, real-time speech-to-text. - Mistral Small 4: Fast, efficient, general-purpose agentic model. - Voxtral TTS: Realistic customizable text-to-speech with streaming output.

Mistral for Developers

28,052 görüntüleme • 7 ay önce

Voxtral TTS is built for global applications supporting 9 languages and powering voice workflows. ✅ Full audio intelligence: Works with Voxtral Transcribe for end-to-end speech-to-speech, or plugs into any STT + LLM stack. ✅ Built for business: From customer support to real-time translation, it’s the output layer that passes the human test. 🎥 See it in action:
1:27

Voxtral TTS is built for global applications supporting 9 languages and powering voice workflows. ✅ Full audio intelligence: Works with Voxtral Transcribe for end-to-end speech-to-speech, or plugs into any STT + LLM stack. ✅ Built for business: From customer support to real-time translation, it’s the output layer that passes the human test. 🎥 See it in action:

Mistral AI

28,284 görüntüleme • 7 ay önce

This is what happens when you plug LLMs into voice assistants, instead of a decade of handwritten rules. This video dissects Voxtral (a family of OSS speech models) and the foundational work behind it (audio tokenization, semantic/acoustic disaggregation, etc). Thank you Mistral AI for your collaboration and for your detailed technical reports in an increasingly opaque industry! 00:00 Intro 01:03 Modular vs end-to-end speech models 03:30 Speech-to-Text 06:07 Delayed Streams Modeling (DSM) 09:41 Whisper Streaming 10:33 Voxtral Realtime 13:07 Voxtral Text-to-Speech 14:28 Throwback: WaveNet 15:24 Audio tokenization 20:39 The Voxtral Codec 21:49 Back to Voxtral TTS 25:30 Outro
26:36

This is what happens when you plug LLMs into voice assistants, instead of a decade of handwritten rules. This video dissects Voxtral (a family of OSS speech models) and the foundational work behind it (audio tokenization, semantic/acoustic disaggregation, etc). Thank you Mistral AI for your collaboration and for your detailed technical reports in an increasingly opaque industry! 00:00 Intro 01:03 Modular vs end-to-end speech models 03:30 Speech-to-Text 06:07 Delayed Streams Modeling (DSM) 09:41 Whisper Streaming 10:33 Voxtral Realtime 13:07 Voxtral Text-to-Speech 14:28 Throwback: WaveNet 15:24 Audio tokenization 20:39 The Voxtral Codec 21:49 Back to Voxtral TTS 25:30 Outro

Julia Turc

298,510 görüntüleme • 4 ay önce

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices
1:21

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices

Mistral AI

953,789 görüntüleme • 7 ay önce

Presenting MetaVoice-1B, a 1.2B parameter base model for TTS (text-to-speech). * Emotional speech in English * Voice cloning with fine-tuning * Zero-shot cloning for American & British voices * Support for long-form synthesis
1:54

Presenting MetaVoice-1B, a 1.2B parameter base model for TTS (text-to-speech). * Emotional speech in English * Voice cloning with fine-tuning * Zero-shot cloning for American & British voices * Support for long-form synthesis

Familiar

111,865 görüntüleme • 2 yıl önce

Introducing Voxtral 3: state-of-the-art Transcribe and Speech models powering Mistral’s new voice agents API. 🧵
2:08

Introducing Voxtral 3: state-of-the-art Transcribe and Speech models powering Mistral’s new voice agents API. 🧵

Mistral AI

282,559 görüntüleme • 19 gün önce

for those who hate reading, demo video of the final speech to speech pipeline, featuring Armin Ronacher ⇌. pretty solid for < 9gb of unified RAM use for STT, TTS, and LLM combined.
1:45

for those who hate reading, demo video of the final speech to speech pipeline, featuring Armin Ronacher ⇌. pretty solid for < 9gb of unified RAM use for STT, TTS, and LLM combined.

Mario Zechner

12,852 görüntüleme • 5 ay önce

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!
1:17

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!

Andi Marafioti

123,084 görüntüleme • 2 ay önce

Our latest speech-to-speech model is faster, more accurate, and excels at function calling. Watch @promptshant and Brian Fioca build a realtime voice agent that can search the web and hand off tasks to reasoning models with full context.
1:58

Our latest speech-to-speech model is faster, more accurate, and excels at function calling. Watch @promptshant and Brian Fioca build a realtime voice agent that can search the web and hand off tasks to reasoning models with full context.

OpenAI Developers

81,863 görüntüleme • 1 yıl önce

We made the first AI model you can have a realistic conversation with that was trained entirely on speech! Dessa is a full-duplex speech-to-speech model that you can run on consumer GPUs. This is a GPT3 moment for speech.
3:25

We made the first AI model you can have a realistic conversation with that was trained entirely on speech! Dessa is a full-duplex speech-to-speech model that you can run on consumer GPUs. This is a GPT3 moment for speech.

Ragavan Thurairatnam

4,647,387 görüntüleme • 16 gün önce

Introducing Voxtral WebGPU: Real-time speech transcription entirely in your browser. This demo runs Voxtral-Mini-4B, a powerful streaming ASR model from Mistral AI, locally on WebGPU. The model supports 13 languages and is capable of <500 ms latency. Fully private. Zero cost.
0:39

Introducing Voxtral WebGPU: Real-time speech transcription entirely in your browser. This demo runs Voxtral-Mini-4B, a powerful streaming ASR model from Mistral AI, locally on WebGPU. The model supports 13 languages and is capable of <500 ms latency. Fully private. Zero cost.

Xenova

94,623 görüntüleme • 8 ay önce

What if voice AI could understand context, intent, and emotion — not just words? 🎙️ Meet Seed Speech 2.0. A new speech AI stack designed for natural, expressive conversations — with major upgrades in both text-to-speech (TTS) and speech recognition (ASR). Built for developers and creators who want natural conversational speech. What’s inside: - Natural and expressive speech generation - Prompt-controlled emotion and tone - High-accuracy multilingual recognition - Strong contextual reasoning Built for: 🎧 Voice content creation 🤖 AI assistants ☎️ Customer service 🎬 Dubbing & subtitling 📊 Audio-video analysis Voice AI built for expression, accuracy, and understanding. Try it now → Read more about Seed Speech here → #SeedSpeech #VoiceAI #SpeechAI #AI
0:30

What if voice AI could understand context, intent, and emotion — not just words? 🎙️ Meet Seed Speech 2.0. A new speech AI stack designed for natural, expressive conversations — with major upgrades in both text-to-speech (TTS) and speech recognition (ASR). Built for developers and creators who want natural conversational speech. What’s inside: - Natural and expressive speech generation - Prompt-controlled emotion and tone - High-accuracy multilingual recognition - Strong contextual reasoning Built for: 🎧 Voice content creation 🤖 AI assistants ☎️ Customer service 🎬 Dubbing & subtitling 📊 Audio-video analysis Voice AI built for expression, accuracy, and understanding. Try it now → Read more about Seed Speech here → #SeedSpeech #VoiceAI #SpeechAI #AI

BytePlus

19,877 görüntüleme • 7 ay önce

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️
4:29

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️

Prince Canuma

164,026 görüntüleme • 8 ay önce

Free speech is not for the speech you like or for the people you like. Free speech is for the speech that you hate of the people you despise. If you're for free speech, you have to be for their free speech, or else you're not for free speech. You're for the government breaking the constitution. Full Video:
1:16

Free speech is not for the speech you like or for the people you like. Free speech is for the speech that you hate of the people you despise. If you're for free speech, you have to be for their free speech, or else you're not for free speech. You're for the government breaking the constitution. Full Video:

Jimmy Dore

69,714 görüntüleme • 1 yıl önce

Build a real-time AI Voice Agent that can actually sell. In less than 30 minutes, you'll learn how to build a sophisticated real-time voice sales agent that can have natural conversations with potential customers. Includes a full code notebook, recorded tutorial, and docs The stack: ⚡ fast inference 🎙 LiveKit for real-time audio/video infrastructure 🗣 realistic speech synthesis (STT/TTS)
0:30

Build a real-time AI Voice Agent that can actually sell. In less than 30 minutes, you'll learn how to build a sophisticated real-time voice sales agent that can have natural conversations with potential customers. Includes a full code notebook, recorded tutorial, and docs The stack: ⚡ fast inference 🎙 LiveKit for real-time audio/video infrastructure 🗣 realistic speech synthesis (STT/TTS)

Sarah Chieng

452,438 görüntüleme • 1 yıl önce

What a week for open audio models! 🕺 💃 I demo: 🗣️ Voxtral 4B TTS from Mistral AI 🎙️ Transcribe 2B from Cohere 🏭 and how to run a batch transcribe job in 1 line of CLI using Daniel van Strien uv script links below
4:39

What a week for open audio models! 🕺 💃 I demo: 🗣️ Voxtral 4B TTS from Mistral AI 🎙️ Transcribe 2B from Cohere 🏭 and how to run a batch transcribe job in 1 line of CLI using Daniel van Strien uv script links below

Jeff Boudier 🤗

32,977 görüntüleme • 7 ay önce

Start experimenting with Voxtral Mini Transcribe 2 in the new audio playground on Mistral Studio. Upload files, toggle diarization, add context bias, and get instant transcriptions.
0:55

Start experimenting with Voxtral Mini Transcribe 2 in the new audio playground on Mistral Studio. Upload files, toggle diarization, add context bias, and get instant transcriptions.

Mistral AI

47,793 görüntüleme • 9 ay önce

SwiftUI: You can chain free and open-source STT, TTS, and LLMs to build voice/vision apps with Core AI. Here is a demo I was experimenting with: - Whisper: Speech-to-text - Kokoro-82M: Text-to-speech - Qwen3-VL 2B: Vision I converted Pocket TTS and VibeVoice to Core AI models and tried with Gemma 4. All is working great.
0:36

SwiftUI: You can chain free and open-source STT, TTS, and LLMs to build voice/vision apps with Core AI. Here is a demo I was experimenting with: - Whisper: Speech-to-text - Kokoro-82M: Text-to-speech - Qwen3-VL 2B: Vision I converted Pocket TTS and VibeVoice to Core AI models and tried with Gemma 4. All is working great.

Amos Gyamfi

80,976 görüntüleme • 4 ay önce

An ‘interactive speech’ from Parliament. A first ? 👉🏻5 of 10 people in rural India lack internet access so they can’t watch speech 👉🏻Yet MGNREGA workers are forced to go online for wages CLIP from my speech on Budget 2026. Parliament 👇🏻 Full speech:
1:06

An ‘interactive speech’ from Parliament. A first ? 👉🏻5 of 10 people in rural India lack internet access so they can’t watch speech 👉🏻Yet MGNREGA workers are forced to go online for wages CLIP from my speech on Budget 2026. Parliament 👇🏻 Full speech:

Derek O'Brien | ডেরেক ও'ব্রায়েন

30,151 görüntüleme • 8 ay önce

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.
0:37

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.

ElevenLabs

113,343 görüntüleme • 2 yıl önce