Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Microsoft have announced their AI "VALL-E" Using a 3-second sample of human speech, it can generate super-high-quality text-to-text speech from the same voice. Even emotional range and acoustic environment of the sample data can be reproduced. Here are some examples.

Del

37,970 subscribers

597,011 views • 3 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Comments

Del Walker3 years ago

At the moment VALL-E can only read, not necessarily PERFORM with the emotional, tonal and pacing range of a voice actor. However, much of the audiobook industry relies on a lot of junior voice actor talent that will undoubtedly feel the brunt of this first.

Kevin Nash 💙3 years ago

This is terrifying thinking about scam callers getting their hands on this.

Del Walker3 years ago

I can honestly seeing the foreign accent scam stereotype disappearing completely once you match this tech with a good script.

RJ Palmer3 years ago

soon the only way to know if a person is real that you're talking to will be physically in person.

waterlilyslim3 years ago

Input: I like turtles Output: I'm calling about your vehicle's extended warranty

Ellski the Artist oder auch einfach nur Elly3 years ago

How, in the name of god or whoever else, did anyone think: „yup. That’s a good idea. Surely no one will use this with ill intentions.“

Christina Kraus3 years ago

What use does this even have except for scam and impersonation purposes? Why don't we focus on AI where it actually helps humanity? Why are we getting AI image generators and voice imitation? That's literally the last thing we need.

Illui3 years ago

Oh ye, this is definitely not going to open a whole can of worms of exploitation, identity thief and fraud ;)

The_Godless_Queen3 years ago

Remember folks, just because is “the new technology” and “technological advancement” doesn’t mean is good or it should be embraced

Andrea3 years ago

@chrisatlatenite Wait till we are all getting scam phone calls from our own mums

Related Videos

High quality AI generated talking heads are coming! GAIA can generate talking avatars from a single portrait image and speech clip. It even supports text prompts like `sad`, `open mouth` or `surprise` to guide video generation. Crazy times ahead 🤯

High quality AI generated talking heads are coming! GAIA can generate talking avatars from a single portrait image and speech clip. It even supports text prompts like `sad`, `open mouth` or `surprise` to guide video generation. Crazy times ahead 🤯

Dreaming Tulpa 🥓👑

660,226 views • 2 years ago

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

Vaibhav (VB) Srivastav

299,066 views • 1 year ago

This is the best open source text-to-speech I’ve ever heard. It’s free and you can run it locally. It’s also unrestricted as you can give it any voice sample. In this video I have it speak as Peter Griffin, Trump, Jarvis, & Scarjo’s voice from HER. More in thread 👇

This is the best open source text-to-speech I’ve ever heard. It’s free and you can run it locally. It’s also unrestricted as you can give it any voice sample. In this video I have it speak as Peter Griffin, Trump, Jarvis, & Scarjo’s voice from HER. More in thread 👇

Blaine Brown

343,292 views • 1 year ago

AI Text-to-Music has arrived. MusicLM is a model by Google Research that generates high-fidelity music from text descriptions. Basically just enter some text and it will create the music. 🤯 Attached video shows sample text prompts and AI generated music.

AI Text-to-Music has arrived. MusicLM is a model by Google Research that generates high-fidelity music from text descriptions. Basically just enter some text and it will create the music. 🤯 Attached video shows sample text prompts and AI generated music.

Dave Lee

72,546 views • 3 years ago

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.

AK

429,355 views • 3 years ago

🎧 AI-read audiobooks are about to explode. Apple is now allowing authors to convert their books into audio using text-to-speech, and the audio quality is insane: Here's some samples from Apple. Details below ↓

🎧 AI-read audiobooks are about to explode. Apple is now allowing authors to convert their books into audio using text-to-speech, and the audio quality is insane: Here's some samples from Apple. Details below ↓

AI Breakfast

146,364 views • 3 years ago

🚨 BREAKING: Text to Speech is coming to Runway “Text to Speech allows you to generate voice overs and dialogue in a number of different voices, all within Runway.” It’s now available to all CPP members. Here’s a quick preview including sample audio I generated, featuring a quote from Steve Jobs who once said, “The most powerful person in the world is the storyteller. The storyteller sets the vision, values and agenda of an entire generation that is to come.”

🚨 BREAKING: Text to Speech is coming to Runway “Text to Speech allows you to generate voice overs and dialogue in a number of different voices, all within Runway.” It’s now available to all CPP members. Here’s a quick preview including sample audio I generated, featuring a quote from Steve Jobs who once said, “The most powerful person in the world is the storyteller. The storyteller sets the vision, values and agenda of an entire generation that is to come.”

David JW Villalva

27,264 views • 2 years ago

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,934 views • 1 year ago

BUILD 🔥: Microsoft is preparing new image and voice models for the announcement on June 2. > MAI Voice 2, a multilingual model supporting 15 news languages and a wider range of emotional spectrum (check voice samples in the article) > MAI Transcribe 1.5, a new model for speech-to-text use cases. > MAI Image 2.5, already announced last week, is now available on LM Arena in preview. Compared to MAI Image 2, it supports file uploads and can be used for image editing.

BUILD 🔥: Microsoft is preparing new image and voice models for the announcement on June 2. > MAI Voice 2, a multilingual model supporting 15 news languages and a wider range of emotional spectrum (check voice samples in the article) > MAI Transcribe 1.5, a new model for speech-to-text use cases. > MAI Image 2.5, already announced last week, is now available on LM Arena in preview. Compared to MAI Image 2, it supports file uploads and can be used for image editing.

🚨 AI News | TestingCatalog

46,535 views • 4 months ago

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

Karan Goel

7,128,131 views • 3 months ago

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 views • 3 years ago

One of the most impressive AI demo I've seen. This is the future of customer service. Agents that can understand text, speech, images and even live video. Soon to be all open-source.

One of the most impressive AI demo I've seen. This is the future of customer service. Agents that can understand text, speech, images and even live video. Soon to be all open-source.

Lior Alexander

271,150 views • 1 year ago

3D AI is leveling up! Rodin 3D AI can create stunning, high-quality 3D models from just text or image inputs. And with its latest update, it can even generate 8K HDRI textures to bring your models to life. Check out the link in the comments!

3D AI is leveling up! Rodin 3D AI can create stunning, high-quality 3D models from just text or image inputs. And with its latest update, it can even generate 8K HDRI textures to bring your models to life. Check out the link in the comments!

el.cine

46,061 views • 1 year ago

I spent the last 2 weeks building the ultimate AI assistant for spreadsheets. It can: - Autofill entire sheets just based off context - Generate formulas and filter rows using text commands - Directly integrate GPT-3 into your sheets And more… Here are a few examples: (1/8)

I spent the last 2 weeks building the ultimate AI assistant for spreadsheets. It can: - Autofill entire sheets just based off context - Generate formulas and filter rows using text commands - Directly integrate GPT-3 into your sheets And more… Here are a few examples: (1/8)

Mehran Jalali

1,143,356 views • 3 years ago

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

Melvin Vivas

27,722 views • 1 year ago

I've just tested the EVI 3 speech-to-speech model by Hume Hume AI, and this is the most realistic voice cloning I’ve heard. It doesn’t just clone the sound of your voice — it captures your tone, your speech patterns, your personality. Feels like your voice is thinking in real time. What’s wild is: — It’s not text-to-speech. It’s speech-to-speech. — You speak → it clones your voice + vibe — And it’s compatible with LLMs like Groq, Anthropic, Deepseek - Try talking to a voice clone: - Want more control? Customize it here: This changes how we’ll interact with AI voices. No more robotic replies — these sound human.

I've just tested the EVI 3 speech-to-speech model by Hume Hume AI, and this is the most realistic voice cloning I’ve heard. It doesn’t just clone the sound of your voice — it captures your tone, your speech patterns, your personality. Feels like your voice is thinking in real time. What’s wild is: — It’s not text-to-speech. It’s speech-to-speech. — You speak → it clones your voice + vibe — And it’s compatible with LLMs like Groq, Anthropic, Deepseek - Try talking to a voice clone: - Want more control? Customize it here: This changes how we’ll interact with AI voices. No more robotic replies — these sound human.

Aaliya

28,046 views • 1 year ago

HoloDreamer can generate enclosed 3D scenes from text descriptions! It does so by first creating a high-quality equirectangular panorama and then rapidly reconstructing the 3D scene using 3D Gaussian Splatting. Links ⬇️

HoloDreamer can generate enclosed 3D scenes from text descriptions! It does so by first creating a high-quality equirectangular panorama and then rapidly reconstructing the 3D scene using 3D Gaussian Splatting. Links ⬇️

Dreaming Tulpa 🥓👑

59,412 views • 2 years ago

BREAKING: MICROSOFT JUST ANNOUNCED TO BAN ITS OWN ENGINEERS FROM USING AI DUE TO THE COST OF USING IT. VP OF NVIDIA SAID, “THE COST OF AI FOR MY TEAM WAS MORE THAN HUMANS” “AI CAN COST MORE THAN HUMAN WORKERS NOW”

BREAKING: MICROSOFT JUST ANNOUNCED TO BAN ITS OWN ENGINEERS FROM USING AI DUE TO THE COST OF USING IT. VP OF NVIDIA SAID, “THE COST OF AI FOR MY TEAM WAS MORE THAN HUMANS” “AI CAN COST MORE THAN HUMAN WORKERS NOW”

Vivek Sen

17,500,452 views • 4 months ago

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.

How to build the world's fastest voice AI bot: - Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster. - Route audio over the internet using WebRTC and edge networking. - Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!) Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms. We used Deepgram's STT and TTS for this bot, and everything is hosted on cerebriumai's serverless GPU infrastructure.

kwindla

282,869 views • 2 years ago