Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️

AI at Meta

810,839 subscribers

326,826 Aufrufe • vor 3 Jahren •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von TDM (e/λ)

TDM (e/λ)vor 3 Jahren

How does Meta keep on winning?

Profilbild von Octavia

Octaviavor 3 Jahren

It’s exciting to see the development of AI moving at a rapid pace Get ready for Octavia AI 🔥

Profilbild von Carlos Alarcón

Carlos Alarcónvor 3 Jahren

soon on @huggingface ?

Profilbild von Leo

Leovor 3 Jahren

Could you make these tools into proper modules, preferably ones that install CLI scripts, like whisper does? Checking out a bunch of repositories and downloading model files, and setting PYTHONPATH manually isn't very user friendly...

Profilbild von Kody Young / 💬 Your AI Interpreter

Kody Young / 💬 Your AI Interpretervor 3 Jahren

Wrote about the implications and breakdown of it here, if anyone wants to dive in deeper:

Profilbild von SFT Protocol

SFT Protocolvor 3 Jahren

Wow! This is truly amazing. Your Massively Multilingual Speech project is making great strides toward breaking language barriers. Can't wait to see the impact it has on multilingual communication! #AI #LanguageBarrier #Communication.

Profilbild von Serçiya^ سەرچیا

Serçiya^ سەرچیاvor 3 Jahren

I suggest creating a demo page or an app to allow people who speak these languages to easily use the tool. Also, it would be beneficial to integrate it with the NLLB-200. @ylecun

Profilbild von Lasley

Lasleyvor 3 Jahren

Wow

Profilbild von Mark Wick

Mark Wickvor 3 Jahren

Hot to test this? Is it public?

Profilbild von @AI_TwistedHumor

@AI_TwistedHumorvor 3 Jahren

Massive congrats to Meta AI for the MMS project! Supporting 1100 languages... now if we can just work out how to say 'Congrats' in each one. Computer, start with Zulu... #AIProgress #LanguageLearning 🌐🔊

Ähnliche Videos

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

AI at Meta

592,726 Aufrufe • vor 2 Jahren

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,739 Aufrufe • vor 1 Jahr

Thanks to our +1.2M community across +180 countries, we are able to source thousands of languages & dialects. This fuels Silencio Voice AI's unmatched datasets: • Multilingual Conversations • Text to Speech • Speech to Text • Ambient Sounds • Commands and Order. Powering the next wave of AI & Robotics. PS: Keynote opening for everyone today, stay tuned.

Thanks to our +1.2M community across +180 countries, we are able to source thousands of languages & dialects. This fuels Silencio Voice AI's unmatched datasets: • Multilingual Conversations • Text to Speech • Speech to Text • Ambient Sounds • Commands and Order. Powering the next wave of AI & Robotics. PS: Keynote opening for everyone today, stay tuned.

Silencio | Voice Data for AI

13,705 Aufrufe • vor 10 Monaten

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️

Rohan Paul

228,836 Aufrufe • vor 1 Jahr

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.

ElevenLabs

113,343 Aufrufe • vor 2 Jahren

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

Karan Goel

7,006,706 Aufrufe • vor 1 Monat

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:

AssemblyAI

42,980 Aufrufe • vor 1 Jahr

we’re lying on the moon. here are a few models glued together — speech to text (OpenAI whisper) to text (OpenAI gpt-3.5-turbo) to speech () — to recreate samantha from “her” in real-time.

we’re lying on the moon. here are a few models glued together — speech to text (OpenAI whisper) to text (OpenAI gpt-3.5-turbo) to speech () — to recreate samantha from “her” in real-time.

harley turan

506,214 Aufrufe • vor 3 Jahren

Introducing Voicebox, a new breakthrough generative speech system based on Flow Matching, a new method proposed by Meta AI. It can synthesize speech across six languages, perform noise removal, edit content, transfer audio style & more. More details on this work & examples ⬇️

Introducing Voicebox, a new breakthrough generative speech system based on Flow Matching, a new method proposed by Meta AI. It can synthesize speech across six languages, perform noise removal, edit content, transfer audio style & more. More details on this work & examples ⬇️

AI at Meta

445,215 Aufrufe • vor 3 Jahren

Can you accurately transcribe fast speech? Tested ' new Speech-to-Text model (Scribe) with Eminem's "Rap God" (4.28 words/sec!) & it nailed it. Great quality and supports 99+ languages.

Can you accurately transcribe fast speech? Tested ' new Speech-to-Text model (Scribe) with Eminem's "Rap God" (4.28 words/sec!) & it nailed it. Great quality and supports 99+ languages.

Addy Osmani

108,722 Aufrufe • vor 1 Jahr

Grok's new Speech to Text model is SO CHEAP and SO ACCURATE 50¢ to transcribe our last 150 videos. We said AI 1,053 times

Grok's new Speech to Text model is SO CHEAP and SO ACCURATE 50¢ to transcribe our last 150 videos. We said AI 1,053 times

Wes Bos

49,144 Aufrufe • vor 2 Monaten

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

AI Breakfast

326,478 Aufrufe • vor 3 Jahren

Eleven v3 now supports Text to Speech in 41 new languages - bringing the total to over 70. This means you can now reach over 90% of the global population with ElevenLabs.

Eleven v3 now supports Text to Speech in 41 new languages - bringing the total to over 70. This means you can now reach over 90% of the global population with ElevenLabs.

ElevenLabs

57,624 Aufrufe • vor 1 Jahr

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Vaibhav Sisinty

24,529,829 Aufrufe • vor 2 Monaten

Introducing Eleven Multilingual v2: our new AI speech model supporting 28 languages! v2 comes with enhanced conversational capability, higher output quality & the ability to better preserve unique voice characteristics across languages. Read more:

Introducing Eleven Multilingual v2: our new AI speech model supporting 28 languages! v2 comes with enhanced conversational capability, higher output quality & the ability to better preserve unique voice characteristics across languages. Read more:

ElevenLabs

293,366 Aufrufe • vor 2 Jahren

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗

Vaibhav (VB) Srivastav

66,963 Aufrufe • vor 1 Jahr

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices

Mistral AI

939,580 Aufrufe • vor 3 Monaten

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 Aufrufe • vor 3 Jahren

Have you heard? You can now listen to News from Science stories using our new text-to-speech feature. 🎧 Try it out for this story on how built-in safeguards might stop AI from designing bioweapons:

Have you heard? You can now listen to News from Science stories using our new text-to-speech feature. 🎧 Try it out for this story on how built-in safeguards might stop AI from designing bioweapons:

Science Magazine

27,118 Aufrufe • vor 1 Jahr