Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Muse Voice Transcribe is MSL's first real-time audio perception model -- rolling out today. SOTA in streaming speech-to-text, it handles speaker diarization, and endpointing natively in a single model.

Mark Zuckerberg

2,379,751 subscribers

1,070,706 views • 9 days ago •via X (Twitter)

Science & Technology Education News & Politics

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Introducing Gemini 3.5 Transcribe, our latest speech-to-text model—available in two distinct model strings: ⚡ Real-time streaming: gemini-3.5-transcribe-live-preview 📁 Pre-recorded audio processing: gemini-3.5-transcribe-preview Let's talk through them 🧵↓

Introducing Gemini 3.5 Transcribe, our latest speech-to-text model—available in two distinct model strings: ⚡ Real-time streaming: gemini-3.5-transcribe-live-preview 📁 Pre-recorded audio processing: gemini-3.5-transcribe-preview Let's talk through them 🧵↓

Google Cloud Tech

28,575 views • 15 days ago

Say hello to GPT-4o, our new flagship model which can reason across audio, vision, and text in real time: Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks.

Say hello to GPT-4o, our new flagship model which can reason across audio, vision, and text in real time: Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks.

OpenAI

22,826,709 views • 2 years ago

introducing Gemini 3.5 Transcribe: our most precise speech-to-text model yet, designed for intelligent voice interactions this new model: - seamlessly handles self-corrections - removes filler words to deliver clean, formatted text - understands your natural intent and speaking style - captures audio, even in noisy environments - supports 85+ languages, including regional accents and diverse dialects try it today in the Gemini API and in AI Studio

introducing Gemini 3.5 Transcribe: our most precise speech-to-text model yet, designed for intelligent voice interactions this new model: - seamlessly handles self-corrections - removes filler words to deliver clean, formatted text - understands your natural intent and speaking style - captures audio, even in noisy environments - supports 85+ languages, including regional accents and diverse dialects try it today in the Gemini API and in AI Studio

Google AI Studio

236,755 views • 15 days ago

Universal-Streaming is our real-time speech-to-text model — built for voice agents with ~300ms latency, high accuracy, unlimited concurrency, and just $0.15/hour. 👉 Test it live in the playground →

Universal-Streaming is our real-time speech-to-text model — built for voice agents with ~300ms latency, high accuracy, unlimited concurrency, and just $0.15/hour. 👉 Test it live in the playground →

AssemblyAI

1,017,880 views • 1 year ago

Introducing Gemini 3.5 Transcribe, our most precise speech-to-text model yet, designed for intelligent voice interactions! 💬 Available for both real-time speech and audio file transcription, with support for: 💡 smart transcription: rewrite and format text based on intent. 🌍 support for 85+ languages, regional accents and diverse dialects. 📚 custom vocabulary: recognize jargon and specific spellings. 💯 high precision in alphanumerics. 🫂 multi-speaker identification.

Introducing Gemini 3.5 Transcribe, our most precise speech-to-text model yet, designed for intelligent voice interactions! 💬 Available for both real-time speech and audio file transcription, with support for: 💡 smart transcription: rewrite and format text based on intent. 🌍 support for 85+ languages, regional accents and diverse dialects. 📚 custom vocabulary: recognize jargon and specific spellings. 💯 high precision in alphanumerics. 🫂 multi-speaker identification.

Thor 雷神 ⚡️

22,696 views • 15 days ago

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

Santiago

66,420 views • 1 year ago

MICROSOFT OPEN SOURCED A 7B PARAMETER MODEL THAT TRANSCRIBES 60 MINUTES OF AUDIO IN A SINGLE PASS and it's completely free VIBEVOICE ASR no chunking, no context loss, full speaker diarization baked in not just speech to text..not a basic wrapper who spoke, when they spoke, exactly what they said..all in one shot and it handles the hard stuff too..50+ languages, custom hotwords, long form audio that breaks every other tool the model doesn't know what "context window" means apparently Available on macOS and Windows right now. Free to use. Free to fine tune. Free to build on.

MICROSOFT OPEN SOURCED A 7B PARAMETER MODEL THAT TRANSCRIBES 60 MINUTES OF AUDIO IN A SINGLE PASS and it's completely free VIBEVOICE ASR no chunking, no context loss, full speaker diarization baked in not just speech to text..not a basic wrapper who spoke, when they spoke, exactly what they said..all in one shot and it handles the hard stuff too..50+ languages, custom hotwords, long form audio that breaks every other tool the model doesn't know what "context window" means apparently Available on macOS and Windows right now. Free to use. Free to fine tune. Free to build on.

Rahul

1,375,224 views • 4 months ago

Better/faster/cheaper voice AI turn detection with Gemini 2.0 The code that determines when the agent should respond to the user is some of the most important code in your voice AI agent. The technical terms for this job are "turn detection" or "phrase endpointing." If the voice AI responds before the user has finished their thought, the conversation is choppy and unproductive. If the AI waits too long, the conversation is slow and frustrating. There are a number of ways to approach this. You can: 1. Use a fast "voice activity detection" model to detect pauses in speech. Respond when the user pauses. 2. Use a specialized phrase endpointing model that operates on transcribed text, pattern-matching on text semantics. 3. Train a specialized phrase endpointing model that operates directly on audio. 4. Leverage the native audio capabilities of a SOTA LLM like Gemini 2.0. We've benchmarked all four of these, and Gemini 2.0 currently beats other approaches. Using Gemini is also cheaper than transcribing the audio separately using a transcription service or model. Here's a short video showing Gemini phrase endpointing in two scenarios. First, correctly handling pauses in natural conversation. Second, requesting a phone number (which is a common activity in a use case like customer support). You can see the Completeness check lines in the terminal output, printed each time Gemini processes a chunk of audio.

Better/faster/cheaper voice AI turn detection with Gemini 2.0 The code that determines when the agent should respond to the user is some of the most important code in your voice AI agent. The technical terms for this job are "turn detection" or "phrase endpointing." If the voice AI responds before the user has finished their thought, the conversation is choppy and unproductive. If the AI waits too long, the conversation is slow and frustrating. There are a number of ways to approach this. You can: 1. Use a fast "voice activity detection" model to detect pauses in speech. Respond when the user pauses. 2. Use a specialized phrase endpointing model that operates on transcribed text, pattern-matching on text semantics. 3. Train a specialized phrase endpointing model that operates directly on audio. 4. Leverage the native audio capabilities of a SOTA LLM like Gemini 2.0. We've benchmarked all four of these, and Gemini 2.0 currently beats other approaches. Using Gemini is also cheaper than transcribing the audio separately using a transcription service or model. Here's a short video showing Gemini phrase endpointing in two scenarios. First, correctly handling pauses in natural conversation. Second, requesting a phone number (which is a common activity in a use case like customer support). You can see the Completeness check lines in the terminal output, printed each time Gemini processes a chunk of audio.

kwindla

64,190 views • 1 year ago

This is "Whisper.cpp" a local lightweight AI model that can be run on mobile phones to convert speech to text with real time streaming

This is "Whisper.cpp" a local lightweight AI model that can be run on mobile phones to convert speech to text with real time streaming

Abdelhamed M.

143,177 views • 1 year ago

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

AI at Meta

592,782 views • 3 years ago

Real-time transcription just got a significant upgrade. Universal-3-Pro is now available for streaming — bringing AssemblyAI's most accurate speech model to live audio for the first time. Developers building voice agents, live captioning tools, and real-time analytics pipelines now get three things they've been asking for: 🔹 Best-in-class word error and entity detection across streaming ASR benchmarks 🔹 Real-time speaker labels — know who said what, as it happens 🔹 Superior entity detection for names, places, orgs, and specialized terminology in real-time 🔹 Code-switching and global language coverage built-in

Real-time transcription just got a significant upgrade. Universal-3-Pro is now available for streaming — bringing AssemblyAI's most accurate speech model to live audio for the first time. Developers building voice agents, live captioning tools, and real-time analytics pipelines now get three things they've been asking for: 🔹 Best-in-class word error and entity detection across streaming ASR benchmarks 🔹 Real-time speaker labels — know who said what, as it happens 🔹 Superior entity detection for names, places, orgs, and specialized terminology in real-time 🔹 Code-switching and global language coverage built-in

AssemblyAI

15,016 views • 6 months ago

Gemini 3.1 Flash Live just dropped and it's available with LiveKit today. This is the first Gemini 3 native audio model on the Live API. Better instruction following, improved tool calling, reduced speaker drift, and support for 70+ languages. Audio in, audio out. No text conversion in between.

Gemini 3.1 Flash Live just dropped and it's available with LiveKit today. This is the first Gemini 3 native audio model on the Live API. Better instruction following, improved tool calling, reduced speaker drift, and support for 70+ languages. Audio in, audio out. No text conversion in between.

LiveKit

40,277 views • 5 months ago

Today, we’re introducing Gemini 3.5 Transcribe, our most precise speech-to-text model yet, designed for intelligent voice interactions in Google Antigravity. It pairs screen context and chat history, with your permission, to ensure pinpoint transcription accuracy across file names, agent thoughts, and active documents.

Today, we’re introducing Gemini 3.5 Transcribe, our most precise speech-to-text model yet, designed for intelligent voice interactions in Google Antigravity. It pairs screen context and chat history, with your permission, to ensure pinpoint transcription accuracy across file names, agent thoughts, and active documents.

Google Antigravity

141,667 views • 15 days ago

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Vaibhav Sisinty

24,536,356 views • 4 months ago

You can now generate real-time speech that sounds conversational. Microsoft just open-sourced VibeVoice, a real-time text-to-speech system with ~300 ms first audio latency and streaming input. It handles long conversations without falling apart. 𝗧𝗵𝗶𝘀 𝗺𝗼𝗱𝗲𝗹 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗹𝗼𝗻𝗴, 𝗺𝘂𝗹𝘁𝗶-𝘀𝗽𝗲𝗮𝗸𝗲𝗿 𝘀𝗽𝗲𝗲𝗰𝗵. It produces up to 90 minutes of audio. It supports up to four distinct speakers. Turn-taking stays consistent over long sessions. 𝗜𝘁 𝘄𝗼𝗿𝗸𝘀 𝗯𝘆 𝗿𝗲𝗱𝘂𝗰𝗶𝗻𝗴 𝘁𝗶𝗺𝗲 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻. Audio compresses into semantic and acoustic tokens. They run at 7.5 Hz instead of frame-level audio. A language model predicts structure. A diffusion head restores acoustic detail. 𝗜𝘁 𝗮𝗹𝗹𝗼𝘄𝘀 𝗹𝗼𝘄-𝗹𝗮𝘁𝗲𝗻𝗰𝘆 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗮𝘂𝗱𝗶𝗼. The real-time variant streams text incrementally. First speech arrives in ~300 ms. A WebSocket demo shows live generation. The code is MIT-licensed and research-only. The repo already passed 20k GitHub stars.

You can now generate real-time speech that sounds conversational. Microsoft just open-sourced VibeVoice, a real-time text-to-speech system with ~300 ms first audio latency and streaming input. It handles long conversations without falling apart. 𝗧𝗵𝗶𝘀 𝗺𝗼𝗱𝗲𝗹 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗹𝗼𝗻𝗴, 𝗺𝘂𝗹𝘁𝗶-𝘀𝗽𝗲𝗮𝗸𝗲𝗿 𝘀𝗽𝗲𝗲𝗰𝗵. It produces up to 90 minutes of audio. It supports up to four distinct speakers. Turn-taking stays consistent over long sessions. 𝗜𝘁 𝘄𝗼𝗿𝗸𝘀 𝗯𝘆 𝗿𝗲𝗱𝘂𝗰𝗶𝗻𝗴 𝘁𝗶𝗺𝗲 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻. Audio compresses into semantic and acoustic tokens. They run at 7.5 Hz instead of frame-level audio. A language model predicts structure. A diffusion head restores acoustic detail. 𝗜𝘁 𝗮𝗹𝗹𝗼𝘄𝘀 𝗹𝗼𝘄-𝗹𝗮𝘁𝗲𝗻𝗰𝘆 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗮𝘂𝗱𝗶𝗼. The real-time variant streams text incrementally. First speech arrives in ~300 ms. A WebSocket demo shows live generation. The code is MIT-licensed and research-only. The repo already passed 20k GitHub stars.

Lior Alexander

61,122 views • 7 months ago

Today we’re introducing Gemini 3.5 Transcribe, our latest transcription model built for incredibly precise, smart dictation across your favorite apps and devices. Remember when traditional speech-to-text meant shouting over background noise, constantly hitting backspace to fix misspelled words, and manually deleting every "um" and "uh"? Those days are over. Gemini 3.5 Transcribe isn't just dictation — it’s active intelligence with precise, context-aware speech-to-text support in 85+ languages. The model automatically filters out filler words, formats unstructured speech, and even pairs with your screen context to execute voice commands. Watch as Gemini 3.5 Transcribe removes filler words and uses multimodal capabilities to seamlessly turn messy voice input and local files into a polished email draft.

Today we’re introducing Gemini 3.5 Transcribe, our latest transcription model built for incredibly precise, smart dictation across your favorite apps and devices. Remember when traditional speech-to-text meant shouting over background noise, constantly hitting backspace to fix misspelled words, and manually deleting every "um" and "uh"? Those days are over. Gemini 3.5 Transcribe isn't just dictation — it’s active intelligence with precise, context-aware speech-to-text support in 85+ languages. The model automatically filters out filler words, formats unstructured speech, and even pairs with your screen context to execute voice commands. Watch as Gemini 3.5 Transcribe removes filler words and uses multimodal capabilities to seamlessly turn messy voice input and local files into a polished email draft.

Google AI

114,490 views • 15 days ago

Google just launched Gemini 3.5 Transcribe, its most precise speech-to-text model yet. I built this app using Gemini 3.5 Transcribe and have been playing with the model for quite some time. Thanks to the Google DeepMind team for the early access. It’s already available in the Gemini API, AI Studio, and Antigravity, and is now in public preview.

Google just launched Gemini 3.5 Transcribe, its most precise speech-to-text model yet. I built this app using Gemini 3.5 Transcribe and have been playing with the model for quite some time. Thanks to the Google DeepMind team for the early access. It’s already available in the Gemini API, AI Studio, and Antigravity, and is now in public preview.

AshutoshShrivastava

20,167 views • 15 days ago

Introducing Marvis-TTS 🔥🚀 A new local-first TTS model Lucas Newman and I built for efficiency, accessibility, and real-time performance right on consumer devices like Apple Silicon, iPhones, iPads, and more. Traditional TTS models often demand full text inputs or sacrifice real-time capabilities, Marvis flips the script. It streams audio chunks as text is processed, creating a truly conversational experience. No more awkward pauses or unnatural breaks—Marvis handles the entire text context intelligently to deliver coherent, expressive speech. Get started today: > pip install -U mlx-audio

Introducing Marvis-TTS 🔥🚀 A new local-first TTS model Lucas Newman and I built for efficiency, accessibility, and real-time performance right on consumer devices like Apple Silicon, iPhones, iPads, and more. Traditional TTS models often demand full text inputs or sacrifice real-time capabilities, Marvis flips the script. It streams audio chunks as text is processed, creating a truly conversational experience. No more awkward pauses or unnatural breaks—Marvis handles the entire text context intelligently to deliver coherent, expressive speech. Get started today: > pip install -U mlx-audio

Prince Canuma

81,404 views • 1 year ago

Today, we’re launching Gemini 3.5 Transcribe, our new speech-to-text model with sub-second streaming and intelligent post-processing for agent interfaces. 2.6% WER on non-streaming and 4.0% on streaming. Supports 85+ languages, cleans up conversational disfluencies ("um", "ah", and mid-sentence self-corrections), handles alphanumeric tokens (postal codes or IDs), 70% reduction time for the final transcription compared to Chirp 3. - `gemini-3.5-transcribe-live`: Sub-second, bidirectional streaming via the Gemini Live API - `gemini-3.5-transcribe`: Recorded processing via the Interactions API with word-level timestamps and speaker attribution (up to 3 speakers). Having access to Transcribe has made me use voice input way more than I ever did before. The native post-processing is incredible. It knows you mean `.json` and not a person named "Jason". I sometimes speak 5 minutes into and it perfectly refactors my instruction to my context. Available today in public preview in Gemini API, Google AI Studio and already in the Google Gemini and Google Antigravity. Try here:

Today, we’re launching Gemini 3.5 Transcribe, our new speech-to-text model with sub-second streaming and intelligent post-processing for agent interfaces. 2.6% WER on non-streaming and 4.0% on streaming. Supports 85+ languages, cleans up conversational disfluencies ("um", "ah", and mid-sentence self-corrections), handles alphanumeric tokens (postal codes or IDs), 70% reduction time for the final transcription compared to Chirp 3. - `gemini-3.5-transcribe-live`: Sub-second, bidirectional streaming via the Gemini Live API - `gemini-3.5-transcribe`: Recorded processing via the Interactions API with word-level timestamps and speaker attribution (up to 3 speakers). Having access to Transcribe has made me use voice input way more than I ever did before. The native post-processing is incredible. It knows you mean `.json` and not a person named "Jason". I sometimes speak 5 minutes into and it perfectly refactors my instruction to my context. Available today in public preview in Gemini API, Google AI Studio and already in the Google Gemini and Google Antigravity. Try here:

Philipp Schmid

15,084 views • 15 days ago