Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🚨 BREAKING: Text to Speech is coming to Runway “Text to Speech allows you to generate voice overs and dialogue in a number of different voices, all within Runway.” It’s now available to all CPP members. Here’s a quick preview including sample audio I generated, featuring a quote from... show more

David JW Villalva

10,247 subscribers

27,262 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

11 Comments

Lachlan Phillips exo/acc 👾2 years ago

@TinfoilTricorn @runwayml These are Elevenlabs voices 100%

Captain HaHaa2 years ago

@runwayml Excellent always good to have everything in the one place

Dave Villalva2 years ago

@Ishouldtrythat1 @runwayml Yes a one stop shop for creative storytelling and video production

Dogan Ural2 years ago

@runwayml Nice, but a bit robotic, I hope they improve it quickly

Dave Villalva2 years ago

@runwayml Yeah just getting started but I rushed a recording before rushing out the door Some sounded more natural @iamneubert was doing some cool multi-lingual experimentations I didn’t even see this coming so I’m now psyched for their audio roadmap

Convey2 years ago

@runwayml Where’d you hear this? Are they talking about it at the conference?

Dave Villalva2 years ago

@runwayml They announced it in their private Discord channel for Creative Partners Program (CPP) members

Benjamin De Kraker2 years ago

@runwayml Any word if this is their own speech model (ie, not ElevenLabs?) Also curious if they will offer this via API?

Dave Villalva2 years ago

@runwayml I’ve seen no mention of partnership with ElevenLabs and it appears to be their own homegrown audio tool They already had some other audio tools

Scottie Wick2 years ago

@runwayml I like the direction this technology is headed. Makes the workflow more efficient. (Prediction - Wav2Lip integration coming next)

Dave Villalva2 years ago

@runwayml Totally been thinking the same thing 🤝 Runway is a one stop shop for generative storytelling

Related Videos

Gemini 2.0 Flash Experimental has the ability to produce native audio in a variety of styles and languages - all from scratch. 🗣️ Here’s how this is different to traditional text-to-speech systems ↓

Gemini 2.0 Flash Experimental has the ability to produce native audio in a variety of styles and languages - all from scratch. 🗣️ Here’s how this is different to traditional text-to-speech systems ↓

Google DeepMind

125,818 views • 1 year ago

Seed Audio 1.0 is now available on Runway for all paid plans. Generate up to 120 seconds of dynamic speech, sound design and music with simple text prompts. Try it now at the link below.

Seed Audio 1.0 is now available on Runway for all paid plans. Generate up to 120 seconds of dynamic speech, sound design and music with simple text prompts. Try it now at the link below.

Runway

32,302 views • 2 months ago

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

AI Breakfast

326,486 views • 3 years ago

Speak with ChatGPT: You can now use voice to engage in a back-and-forth conversation with your assistant. The hyper-realistic text-to-speech model allows you to choose from five different voices. On mobile, opt-in to voice in Settings → New Features on the mobile app.

Speak with ChatGPT: You can now use voice to engage in a back-and-forth conversation with your assistant. The hyper-realistic text-to-speech model allows you to choose from five different voices. On mobile, opt-in to voice in Settings → New Features on the mobile app.

Rowan Cheung

55,915 views • 2 years ago

I made a few updates to the MLX port of F5 TTS over the holiday: - Longform generation is now supported, and text will be split automatically on sentence boundaries for natural-sounding speech. - You can now pipe in text from another process (e.g. a language model) and listen to the speech as each segment is generated. - I implemented RK4 sampling for the ODE, which achieves much better quality with fewer steps — it's now possible to generate speech at 1.6x realtime on my M3 Max. See the video of an example of generating speech in realtime from a language model entirely on-device! 🚀

I made a few updates to the MLX port of F5 TTS over the holiday: - Longform generation is now supported, and text will be split automatically on sentence boundaries for natural-sounding speech. - You can now pipe in text from another process (e.g. a language model) and listen to the speech as each segment is generated. - I implemented RK4 sampling for the ODE, which achieves much better quality with fewer steps — it's now possible to generate speech at 1.6x realtime on my M3 Max. See the video of an example of generating speech in realtime from a language model entirely on-device! 🚀

Lucas Newman

22,998 views • 1 year ago

EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine github: EmotiVoice is a powerful and modern open-source text-to-speech engine. EmotiVoice speaks both English and Chinese, and with over 2000 different voices. The most prominent feature is emotional synthesis, allowing you to create speech with a wide range of emotions, including happy, excited, sad, angry and others.

EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine github: EmotiVoice is a powerful and modern open-source text-to-speech engine. EmotiVoice speaks both English and Chinese, and with over 2000 different voices. The most prominent feature is emotional synthesis, allowing you to create speech with a wide range of emotions, including happy, excited, sad, angry and others.

AK

312,365 views • 2 years ago

Text to Sound Effects is here. Our newest AI Audio model generates sound effects, short instrumental tracks, soundscapes, and a wide variety of character voices, all from a text prompt. Available now for all users. Everyone from content creators, video game developers, to film and television studios, uses sound effects to create rich and immersive content. Now, in addition to AI voiceovers, you can generate all of the sounds you need with just a prompt. Everything you hear in this video was generated by ElevenLabs sound and voice models. In this thread, we shared some additional clips that help show off the range of this new model.

Text to Sound Effects is here. Our newest AI Audio model generates sound effects, short instrumental tracks, soundscapes, and a wide variety of character voices, all from a text prompt. Available now for all users. Everyone from content creators, video game developers, to film and television studios, uses sound effects to create rich and immersive content. Now, in addition to AI voiceovers, you can generate all of the sounds you need with just a prompt. Everything you hear in this video was generated by ElevenLabs sound and voice models. In this thread, we shared some additional clips that help show off the range of this new model.

ElevenLabs

450,763 views • 2 years ago

Here is something completely new you can do in one shot with Runway. This entire video was generated from a single long audio file. Runway analyzed the file, read the audio transcription, and generated one coherent video to bring it all together visually.

Here is something completely new you can do in one shot with Runway. This entire video was generated from a single long audio file. Runway analyzed the file, read the audio transcription, and generated one coherent video to bring it all together visually.

Cristóbal Valenzuela

21,593 views • 1 month ago

MatrixGPT - The state-of-the-art Text to Image AI app 💎An all-in-one #AI solution offering a suite of innovative apps including Text-to-image, Image-to-image, Text-to-speech & Talking face. Let's create art and earn with #MatrixGPT! JOIN US NOW!👉 $MAI

MatrixGPT - The state-of-the-art Text to Image AI app 💎An all-in-one #AI solution offering a suite of innovative apps including Text-to-image, Image-to-image, Text-to-speech & Talking face. Let's create art and earn with #MatrixGPT! JOIN US NOW!👉 $MAI

MatrixGPT

13,534 views • 3 years ago

The most important phone number in sports betting is now (305) 857-2288. That's the number for Gambly Bot text/iMessage, which lets you send screenshots, text, and voice commands to generate one-click betslips. Here's why GamblyBot text is THE way to create betslips 👇

The most important phone number in sports betting is now (305) 857-2288. That's the number for Gambly Bot text/iMessage, which lets you send screenshots, text, and voice commands to generate one-click betslips. Here's why GamblyBot text is THE way to create betslips 👇

Jonathan Bales

149,311 views • 11 months ago

🚨Breaking News🚨 PixVerse 3.5’s text-to-video mode is super versatile and creative. 🎥✨ It’s still in early access, but the potential is huge! Here’s a quick showcase of its text-to-video capabilities. 🔊 Turn up the sound for the full experience! 👇

🚨Breaking News🚨 PixVerse 3.5’s text-to-video mode is super versatile and creative. 🎥✨ It’s still in early access, but the potential is huge! Here’s a quick showcase of its text-to-video capabilities. 🔊 Turn up the sound for the full experience! 👇

Pierrick Chevallier | IA

11,535 views • 1 year ago

OpenAI's S2S preview is polished but it still thinks in steps. Speech → text → model → text → speech. That's not how humans converse. Introducing Hydra. A native speech-to-speech model that doesn't wait for turn-taking, doesn't flatten emotion into text, and doesn't break when you interrupt it mid-sentence. Hydra reasons asynchronously, speaks and listens simultaneously, and preserves emotion because it never leaves the audio domain. It's still in beta, but the shift is obvious. If you want early access, the link is in the comments. Here's a preview of what that looks like -

OpenAI's S2S preview is polished but it still thinks in steps. Speech → text → model → text → speech. That's not how humans converse. Introducing Hydra. A native speech-to-speech model that doesn't wait for turn-taking, doesn't flatten emotion into text, and doesn't break when you interrupt it mid-sentence. Hydra reasons asynchronously, speaks and listens simultaneously, and preserves emotion because it never leaves the audio domain. It's still in beta, but the shift is obvious. If you want early access, the link is in the comments. Here's a preview of what that looks like -

Sudarshan Kamath

329,032 views • 5 months ago

We’re finally breaking the silence 🔊 Meet Higgsfield Audio, our all-in-one AI audio generator. Higgsfield is now the ultimate full-cycle AI content production platform. 🧩 Give a voice to any of your text with 21 voice presets 🧩 Change the voice in any of your videos 🧩 Translate and lip-sync any speech in 10 built-in languages An end-to-end AI workflow under one subscription.

We’re finally breaking the silence 🔊 Meet Higgsfield Audio, our all-in-one AI audio generator. Higgsfield is now the ultimate full-cycle AI content production platform. 🧩 Give a voice to any of your text with 21 voice presets 🧩 Change the voice in any of your videos 🧩 Translate and lip-sync any speech in 10 built-in languages An end-to-end AI workflow under one subscription.

Higgsfield AI 🧩

89,429 views • 5 months ago

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 views • 3 years ago

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

Melvin Vivas

27,722 views • 1 year ago

🌈' The test of the Intro' I would like to give MC voice-over, what do you think? (Now is text-to-speech for the testing)

🌈' The test of the Intro' I would like to give MC voice-over, what do you think? (Now is text-to-speech for the testing)

Mr.Ryu

22,813 views • 1 year ago

Seedream 5.0 Pro is now in Runway. Generate highly detailed images from a prompt or reference, with legible in-image text in up to 14 different languages. Try it now at the link below.

Seedream 5.0 Pro is now in Runway. Generate highly detailed images from a prompt or reference, with legible in-image text in up to 14 different languages. Try it now at the link below.

Runway

21,048 views • 1 month ago

Seedance 2.0 is now on Runway. Use text, image, video or audio as inputs to generate stunning multi-shot video sequences with full sound design and dialogue. Available now on Unlimited plans and Enterprise accounts outside of the US. Get started now at the link below.

Seedance 2.0 is now on Runway. Use text, image, video or audio as inputs to generate stunning multi-shot video sequences with full sound design and dialogue. Available now on Unlimited plans and Enterprise accounts outside of the US. Get started now at the link below.

Runway

188,555 views • 4 months ago

Today, we’re excited to introduce Miso One, the most emotive voice model in the world. Miso One is an 8-billion-parameter text-to-speech model for highly expressive speech generation. It emotes like a human and responds faster than a human, with just 110 milliseconds of latency. We’ve open-sourced the model weights, with API access coming soon. Hear how Miso One sounds in the thread below.

Today, we’re excited to introduce Miso One, the most emotive voice model in the world. Miso One is an 8-billion-parameter text-to-speech model for highly expressive speech generation. It emotes like a human and responds faster than a human, with just 110 milliseconds of latency. We’ve open-sourced the model weights, with API access coming soon. Hear how Miso One sounds in the thread below.

Aoden Teo

5,227,160 views • 2 months ago

I just used Luma's text-to-3D to generate a DeLorean from Back to the Future and placed it in my living room using the Vision Pro! The future is wild. 🤩

I just used Luma's text-to-3D to generate a DeLorean from Back to the Future and placed it in my living room using the Vision Pro! The future is wild. 🤩

Ammaar Reshi

265,563 views • 2 years ago