Loading video...

Video Failed to Load

Go Home

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️

AIatMeta's profile picture

AI at Meta

810,839 subscribers

326,826 views • 3 years ago •via X (Twitter)

Science & TechnologyEducation
LIVE

Anya Rossi• Live Now

Private livecam show

10 Comments

TDM (e/λ)'s profile picture
TDM (e/λ)3 years ago

How does Meta keep on winning?

Octavia's profile picture
Octavia3 years ago

It’s exciting to see the development of AI moving at a rapid pace Get ready for Octavia AI 🔥

Carlos Alarcón's profile picture
Carlos Alarcón3 years ago

soon on @huggingface ?

Leo's profile picture
Leo3 years ago

Could you make these tools into proper modules, preferably ones that install CLI scripts, like whisper does? Checking out a bunch of repositories and downloading model files, and setting PYTHONPATH manually isn't very user friendly...

Kody Young / 💬 Your AI Interpreter's profile picture
Kody Young / 💬 Your AI Interpreter3 years ago

Wrote about the implications and breakdown of it here, if anyone wants to dive in deeper:

SFT Protocol's profile picture
SFT Protocol3 years ago

Wow! This is truly amazing. Your Massively Multilingual Speech project is making great strides toward breaking language barriers. Can't wait to see the impact it has on multilingual communication! #AI #LanguageBarrier #Communication.

Serçiya^ سەرچیا's profile picture
Serçiya^ سەرچیا3 years ago

I suggest creating a demo page or an app to allow people who speak these languages to easily use the tool. Also, it would be beneficial to integrate it with the NLLB-200. @ylecun

Lasley's profile picture
Lasley3 years ago

Wow

Mark Wick's profile picture
Mark Wick3 years ago

Hot to test this? Is it public?

@AI_TwistedHumor's profile picture
@AI_TwistedHumor3 years ago

Massive congrats to Meta AI for the MMS project! Supporting 1100 languages... now if we can just work out how to say 'Congrats' in each one. Computer, start with Zulu... #AIProgress #LanguageLearning 🌐🔊

Related Videos

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️
0:56

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

AI at Meta

592,726 views • 2 years ago

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.
0:36

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,698 views • 1 year ago

Thanks to our +1.2M community across +180 countries, we are able to source thousands of languages & dialects. This fuels Silencio Voice AI's unmatched datasets: • Multilingual Conversations • Text to Speech • Speech to Text • Ambient Sounds • Commands and Order. Powering the next wave of AI & Robotics. PS: Keynote opening for everyone today, stay tuned.
0:57

Thanks to our +1.2M community across +180 countries, we are able to source thousands of languages & dialects. This fuels Silencio Voice AI's unmatched datasets: • Multilingual Conversations • Text to Speech • Speech to Text • Ambient Sounds • Commands and Order. Powering the next wave of AI & Robotics. PS: Keynote opening for everyone today, stay tuned.

Silencio | Voice Data for AI

13,705 views • 9 months ago

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️
1:23

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️

Rohan Paul

228,836 views • 1 year ago

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.
0:37

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.

ElevenLabs

113,343 views • 2 years ago

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.
2:47

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

Karan Goel

7,005,041 views • 28 days ago

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:
1:01

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:

AssemblyAI

42,980 views • 1 year ago

we’re lying on the moon. here are a few models glued together — speech to text (OpenAI whisper) to text (OpenAI gpt-3.5-turbo) to speech () — to recreate samantha from “her” in real-time.
1:18

we’re lying on the moon. here are a few models glued together — speech to text (OpenAI whisper) to text (OpenAI gpt-3.5-turbo) to speech () — to recreate samantha from “her” in real-time.

harley turan

506,214 views • 3 years ago

MyRobot Dev Blog 1/14/26 - Quality of Life I added offline text to speech support (Chatterbox) so you can host your own offline text to speech models.
0:45

MyRobot Dev Blog 1/14/26 - Quality of Life I added offline text to speech support (Chatterbox) so you can host your own offline text to speech models.

MyRobot Dev

42,137 views • 4 months ago

Introducing Voicebox, a new breakthrough generative speech system based on Flow Matching, a new method proposed by Meta AI. It can synthesize speech across six languages, perform noise removal, edit content, transfer audio style & more. More details on this work & examples ⬇️
2:43

Introducing Voicebox, a new breakthrough generative speech system based on Flow Matching, a new method proposed by Meta AI. It can synthesize speech across six languages, perform noise removal, edit content, transfer audio style & more. More details on this work & examples ⬇️

AI at Meta

445,215 views • 3 years ago

Can you accurately transcribe fast speech? Tested ' new Speech-to-Text model (Scribe) with Eminem's "Rap God" (4.28 words/sec!) & it nailed it. Great quality and supports 99+ languages.
1:38

Can you accurately transcribe fast speech? Tested ' new Speech-to-Text model (Scribe) with Eminem's "Rap God" (4.28 words/sec!) & it nailed it. Great quality and supports 99+ languages.

Addy Osmani

108,722 views • 1 year ago

Grok's new Speech to Text model is SO CHEAP and SO ACCURATE 50¢ to transcribe our last 150 videos. We said AI 1,053 times
4:38

Grok's new Speech to Text model is SO CHEAP and SO ACCURATE 50¢ to transcribe our last 150 videos. We said AI 1,053 times

Wes Bos

49,144 views • 2 months ago

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:
0:20

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

AI Breakfast

326,478 views • 3 years ago

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.
0:27

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Vaibhav Sisinty

24,528,901 views • 2 months ago

Eleven v3 now supports Text to Speech in 41 new languages - bringing the total to over 70. This means you can now reach over 90% of the global population with ElevenLabs.
2:37

Eleven v3 now supports Text to Speech in 41 new languages - bringing the total to over 70. This means you can now reach over 90% of the global population with ElevenLabs.

ElevenLabs

57,624 views • 1 year ago

just 🦁 & 👹 sharing the same text-to-speech braincell #lucalive #VoxPopuLIVE
0:40

just 🦁 & 👹 sharing the same text-to-speech braincell #lucalive #VoxPopuLIVE

jeni 🌻🌷

27,925 views • 1 year ago

Introducing Eleven Multilingual v2: our new AI speech model supporting 28 languages! v2 comes with enhanced conversational capability, higher output quality & the ability to better preserve unique voice characteristics across languages. Read more:
1:20

Introducing Eleven Multilingual v2: our new AI speech model supporting 28 languages! v2 comes with enhanced conversational capability, higher output quality & the ability to better preserve unique voice characteristics across languages. Read more:

ElevenLabs

293,366 views • 2 years ago

🗣️ 00:00 Intro 01:32 Audio agents 03:27 Speech-to-text 06:18 Text-to-speech 08:48 Agents SDK Read more in our blog post:
15:09

🗣️ 00:00 Intro 01:32 Audio agents 03:27 Speech-to-text 06:18 Text-to-speech 08:48 Agents SDK Read more in our blog post:

OpenAI Developers

494,109 views • 1 year ago

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗
1:08

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗

Vaibhav (VB) Srivastav

66,963 views • 1 year ago

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices
1:21

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices

Mistral AI

939,580 views • 3 months ago