Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects... show more

AI at Meta

860,576 subscribers

351,956 просмотров • 2 лет назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 10

Фото профиля AI at Meta

AI at Meta2 лет назад

More details, including links to the research paper, model weights and code 👇

Фото профиля $@®+#@|=

$@®+#@|=2 лет назад

Sounds ass ngl

Фото профиля floating point

floating point2 лет назад

Non commercial and poor quality speech? Sad 😔

Фото профиля Leocifer

Leocifer2 лет назад

europe

Фото профиля Tech Dev Notes

Tech Dev Notes2 лет назад

The demo was a bit ...

Фото профиля BensenHsu

BensenHsu2 лет назад

The study introduces S PI R IT -LM, a model that can generate both speech and text. It is based on continuously pre-training a text language model (L LAMA 2) with a combination of text-only, speech-only, and aligned speech-text datasets. S PI R IT -LM performs well on speech and text comprehension tasks, matching or exceeding the performance of previous speech-only and text-only models. It can also learn new tasks in a few-shot setting, both within and across modalities (speech-to-text and text-to-speech). The S PI R IT -LM-E XPRESSIVE version is the first language model that can preserve the sentiment of text and speech prompts both within and across modalities. full paper:

Фото профиля $Q*🍓on Ethereum

$Q*🍓on Ethereum2 лет назад

Everything happening at once

Фото профиля Hamza

Hamza2 лет назад

this preview seems to lack somewhat

Фото профиля Risphere

Risphere2 лет назад

The quality isn't that good.

Фото профиля Qual

Qual2 лет назад

I love you, but this demo was... well, let's just say it had a rough start! At first, I thought my speakers were broken because there was no sound for a few seconds. Then, when the sound finally kicked in, I was like, "Yep, my speakers are definitely broken!"

Похожие видео

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

Karan Goel

7,133,023 просмотров • 4 месяцев назад

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,729 просмотров • 3 лет назад

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

AI at Meta

592,910 просмотров • 3 лет назад

What if voice AI could understand context, intent, and emotion — not just words? 🎙️ Meet Seed Speech 2.0. A new speech AI stack designed for natural, expressive conversations — with major upgrades in both text-to-speech (TTS) and speech recognition (ASR). Built for developers and creators who want natural conversational speech. What’s inside: - Natural and expressive speech generation - Prompt-controlled emotion and tone - High-accuracy multilingual recognition - Strong contextual reasoning Built for: 🎧 Voice content creation 🤖 AI assistants ☎️ Customer service 🎬 Dubbing & subtitling 📊 Audio-video analysis Voice AI built for expression, accuracy, and understanding. Try it now → Read more about Seed Speech here → #SeedSpeech #VoiceAI #SpeechAI #AI

What if voice AI could understand context, intent, and emotion — not just words? 🎙️ Meet Seed Speech 2.0. A new speech AI stack designed for natural, expressive conversations — with major upgrades in both text-to-speech (TTS) and speech recognition (ASR). Built for developers and creators who want natural conversational speech. What’s inside: - Natural and expressive speech generation - Prompt-controlled emotion and tone - High-accuracy multilingual recognition - Strong contextual reasoning Built for: 🎧 Voice content creation 🤖 AI assistants ☎️ Customer service 🎬 Dubbing & subtitling 📊 Audio-video analysis Voice AI built for expression, accuracy, and understanding. Try it now → Read more about Seed Speech here → #SeedSpeech #VoiceAI #SpeechAI #AI

BytePlus

19,877 просмотров • 7 месяцев назад

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️

Today we're sharing new progress on our AI speech work. Our Massively Multilingual Speech (MMS) project has now scaled speech-to-text & text-to-speech to support over 1,100 languages — a 10x increase from previous work. Details + access to new pretrained models ⬇️

AI at Meta

326,946 просмотров • 3 лет назад

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices

🔊Introducing Voxtral TTS: our new frontier open-weight model for natural, expressive, and ultra-fast text-to-speech 🎭Realistic, emotionally expressive speech. 🌍Supports 9 languages and accurately captures diverse dialects. ⚡Very low latency for time-to-first-audio. 🔄Easily adaptable to new voices

Mistral AI

953,789 просмотров • 7 месяцев назад

We pioneered the first ultra-realistic Text to Speech model, and recently launched the world's most accurate Speech to Text model, Scribe. But we're not stopping there. Today, we're taking one small step for man, and one giant leap for man's best friend... with Text to Bark.

We pioneered the first ultra-realistic Text to Speech model, and recently launched the world's most accurate Speech to Text model, Scribe. But we're not stopping there. Today, we're taking one small step for man, and one giant leap for man's best friend... with Text to Bark.

ElevenLabs

291,881 просмотров • 1 год назад

Thanks to our +1.2M community across +180 countries, we are able to source thousands of languages & dialects. This fuels Silencio Voice AI's unmatched datasets: • Multilingual Conversations • Text to Speech • Speech to Text • Ambient Sounds • Commands and Order. Powering the next wave of AI & Robotics. PS: Keynote opening for everyone today, stay tuned.

Thanks to our +1.2M community across +180 countries, we are able to source thousands of languages & dialects. This fuels Silencio Voice AI's unmatched datasets: • Multilingual Conversations • Text to Speech • Speech to Text • Ambient Sounds • Commands and Order. Powering the next wave of AI & Robotics. PS: Keynote opening for everyone today, stay tuned.

Silencio | Voice Data for AI

13,705 просмотров • 1 год назад

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.

AK

429,355 просмотров • 3 лет назад

Today, we're introducing Bulbul V4 Flash, our latest text to speech model built for natural, expressive speech across Indian languages. Flash brings frontier-quality speech into a smaller, more efficient model, preserving the subtle details that make voices sound natural while making speech generation significantly faster. See details in our blog:

Today, we're introducing Bulbul V4 Flash, our latest text to speech model built for natural, expressive speech across Indian languages. Flash brings frontier-quality speech into a smaller, more efficient model, preserving the subtle details that make voices sound natural while making speech generation significantly faster. See details in our blog:

Sarvam

161,274 просмотров • 25 дней назад

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Stripe

18,605 просмотров • 6 месяцев назад

In this demo, we show real-time voice translation between Indian languages. The user can speak in their preferred language, and the system translates and responds in another, preserving both meaning and natural delivery. Speech recognition, translation, and expressive text-to-speech work together seamlessly.

In this demo, we show real-time voice translation between Indian languages. The user can speak in their preferred language, and the system translates and responds in another, preserving both meaning and natural delivery. Speech recognition, translation, and expressive text-to-speech work together seamlessly.

Pratyush Kumar

15,206 просмотров • 8 месяцев назад

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!

Speech-to-speech no longer needs speech-to-text! Until now, our stack was VAD -> STT -> LLM -> TTS. Now it can send audio directly to multimodal LLMs: VAD → MLLM → TTS No STT. The model understands your voice. Now go build better voice agents!

Andi Marafioti

122,929 просмотров • 2 месяцев назад

Cboard is an open source AAC web app that helps people with speech and language disabilities communicate using symbols and text-to-speech. Thanks to its global community of developers, educators, translators, and contributors, Cboard supports accessible communication for people of all ages.

Cboard is an open source AAC web app that helps people with speech and language disabilities communicate using symbols and text-to-speech. Thanks to its global community of developers, educators, translators, and contributors, Cboard supports accessible communication for people of all ages.

GitHub

81,658 просмотров • 2 месяцев назад

Introducing Cekura Bench: the world’s leading Voice AI Benchmarks We benchmarked 50+ models from openAi, Google, Nvidia, xAI, Inworld, Gradium, Assembly, Fish audio and more Now you can: → Compare naturalness, performance and cost → See how the best voice agents are being built. → Find the best fit for what you are building. Our tests cover Speech-to-Speech models, Speech-to-Text models, Text-to-Speech Models, and Voice Agent platforms. We recorded real humans doing tasks in different domains and created the most representative voice benchmark out there! Explore the benchmarks: We would love to collaborate with more teams on these benchmarks. Reach out if you are interested.

Introducing Cekura Bench: the world’s leading Voice AI Benchmarks We benchmarked 50+ models from openAi, Google, Nvidia, xAI, Inworld, Gradium, Assembly, Fish audio and more Now you can: → Compare naturalness, performance and cost → See how the best voice agents are being built. → Find the best fit for what you are building. Our tests cover Speech-to-Speech models, Speech-to-Text models, Text-to-Speech Models, and Voice Agent platforms. We recorded real humans doing tasks in different domains and created the most representative voice benchmark out there! Explore the benchmarks: We would love to collaborate with more teams on these benchmarks. Reach out if you are interested.

Tarush Agarwal

798,909 просмотров • 2 дней назад

Introducing AssemblyAI-TTS-1, our first text-to-speech model. Natural speech for voice agents, streaming in 90 ms, at $15 per million characters. On accuracy, naturalness, and cost together, it’s the best TTS model you can run at scale.

Introducing AssemblyAI-TTS-1, our first text-to-speech model. Natural speech for voice agents, streaming in 90 ms, at $15 per million characters. On accuracy, naturalness, and cost together, it’s the best TTS model you can run at scale.

AssemblyAI

248,329 просмотров • 10 дней назад

I made a few updates to the MLX port of F5 TTS over the holiday: - Longform generation is now supported, and text will be split automatically on sentence boundaries for natural-sounding speech. - You can now pipe in text from another process (e.g. a language model) and listen to the speech as each segment is generated. - I implemented RK4 sampling for the ODE, which achieves much better quality with fewer steps — it's now possible to generate speech at 1.6x realtime on my M3 Max. See the video of an example of generating speech in realtime from a language model entirely on-device! 🚀

I made a few updates to the MLX port of F5 TTS over the holiday: - Longform generation is now supported, and text will be split automatically on sentence boundaries for natural-sounding speech. - You can now pipe in text from another process (e.g. a language model) and listen to the speech as each segment is generated. - I implemented RK4 sampling for the ODE, which achieves much better quality with fewer steps — it's now possible to generate speech at 1.6x realtime on my M3 Max. See the video of an example of generating speech in realtime from a language model entirely on-device! 🚀

Lucas Newman

22,998 просмотров • 1 год назад

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.

Speech to Speech is now available in 29 languages. In November, we launched Speech to Speech, enabling you to transform your voice into another character with full control over emotions, timing, and delivery. Today we're making Speech to Speech multilingual. Use Speech to Speech to get more control than with prompting alone. Simply say it how you want it, choose a target voice, and generate.

ElevenLabs

113,343 просмотров • 2 лет назад

HUMAIN Voice is built for how Arabic speakers actually communicate, across dialects, natural code-switching and everyday conversations. Bringing speech-to-text, text-to-speech and agentic voice interaction into one technology family, it enables more natural voice experiences for enterprises, developers and everyday users. Read more: #HUMAIN #LEAP26 #TheEndOfLimits

HUMAIN Voice is built for how Arabic speakers actually communicate, across dialects, natural code-switching and everyday conversations. Bringing speech-to-text, text-to-speech and agentic voice interaction into one technology family, it enables more natural voice experiences for enterprises, developers and everyday users. Read more: #HUMAIN #LEAP26 #TheEndOfLimits

HUMAIN

21,632 просмотров • 2 месяцев назад

Two Realtime API updates: - You can now build speech-to-speech experiences with five new voices—which are much more expressive and steerable. 🤣🤫🤪 - We're lowering the price by using prompt caching. Cached text inputs are discounted 50% and cached audio inputs are discounted 80%. 📉

Two Realtime API updates: - You can now build speech-to-speech experiences with five new voices—which are much more expressive and steerable. 🤣🤫🤪 - We're lowering the price by using prompt caching. Cached text inputs are discounted 50% and cached audio inputs are discounted 80%. 📉

OpenAI Developers

243,432 просмотров • 2 лет назад