Loading video...

Video Failed to Load

Go Home

HOLY CRAP, a new super tiny 1.6B param voice model just dropped that seems to.. outperform 11labs!? 😵‍💫 From Nari-labs, Dia is an Apache 2.0 voice model, that can generate laughs, sniffs and emotions, copy an existing voice and is effectively real time on larger GPUs:

altryne's profile picture

Alex Volkov

41,240 subscribers

525,803 views • 1 year ago •via X (Twitter)

MusicNews & PoliticsScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

10 Comments

Alex Volkov (Thursd/AI)'s profile picture
Alex Volkov (Thursd/AI)1 year ago

We'll of course cover this on the next @thursdai_pod (thanks for the heads up @rodrimora !)

Alex Volkov (Thursd/AI)'s profile picture
Alex Volkov (Thursd/AI)1 year ago

Here's the links: Demo Page: Github: HF Model: Try It:

Prince Canuma's profile picture
Prince Canuma1 year ago

Coming to MLX Audio this week 🔥 @lllucas and I are on it 🚀

Alex Volkov (Thursd/AI)'s profile picture
Alex Volkov (Thursd/AI)1 year ago

@lllucas can't wait to try this on my mac!

Prince Canuma's profile picture
Prince Canuma1 year ago

The best sniff emotion I have heard so far Other models feel like a sigh

Alex Volkov (Thursd/AI)'s profile picture
Alex Volkov (Thursd/AI)1 year ago

(sniff) the laughter is so dope as well, it comes in juuust before and you can hear the smile on the invisible's AI face as it starts to render the laughter

Jonathan's profile picture
Jonathan1 year ago

Intonation and tone are great, but quality is nowhere near elevenlabs.

Loic's profile picture
Loic1 year ago

Honestly voice generation feels like it has been stuck at the same level for at least a year, not real improvement

Yng_Pepe's profile picture
Yng_Pepe1 year ago

@turing_hamster

Michael Finney's profile picture
Michael Finney1 year ago

This on Pinokio yet @cocktailpeanut ?

Related Videos

Pretty WILD - SoTA open source TTS model that beats ElevenLabs/ Sesame - Dia 1.6B - Apache 2.0 licensed! 🔥 > Ultra realistic voice synthesis > Capable of producing non-verbal sounds - coughing, laughing 💥 > Zero shot Voice Cloning > Real-time TTS synthesis > Can run on your MacBook > Trending #2 on Hugging Face Weights on the Hub and code on GitHub! 🤯
0:27

Pretty WILD - SoTA open source TTS model that beats ElevenLabs/ Sesame - Dia 1.6B - Apache 2.0 licensed! 🔥 > Ultra realistic voice synthesis > Capable of producing non-verbal sounds - coughing, laughing 💥 > Zero shot Voice Cloning > Real-time TTS synthesis > Can run on your MacBook > Trending #2 on Hugging Face Weights on the Hub and code on GitHub! 🤯

Vaibhav (VB) Srivastav

39,165 views • 1 year ago

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗
1:21

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

Vaibhav (VB) Srivastav

298,953 views • 1 year ago

This is the DeepSeek moment for Voice AI. Chatterbox Turbo is an MIT-licensed voice model that beats ElevenLabs Turbo & Cartesia Sonic 3! - <150ms time-to-first-sound - Voice cloning from just 5-second audio - Paralinguistic tags for real human expression 100% open-source.
0:26

This is the DeepSeek moment for Voice AI. Chatterbox Turbo is an MIT-licensed voice model that beats ElevenLabs Turbo & Cartesia Sonic 3! - <150ms time-to-first-sound - Voice cloning from just 5-second audio - Paralinguistic tags for real human expression 100% open-source.

Akshay 🚀

468,456 views • 7 months ago

Today we're releasing ZONOS2, our next-generation real-time TTS model with high-fidelity voice cloning. ZONOS2 is the most expressive open-source TTS model, released under Apache 2.0 and available on Zyphra Cloud on AMD. 🧵
1:30

Today we're releasing ZONOS2, our next-generation real-time TTS model with high-fidelity voice cloning. ZONOS2 is the most expressive open-source TTS model, released under Apache 2.0 and available on Zyphra Cloud on AMD. 🧵

Zyphra

333,345 views • 1 month ago

Today, we’re proud to unveil a pioneering partnership with @Groq. Our Dialog voice model now runs on GroqCloud. This isn't just an upgrade; this is a leap forward that redefines what voice technology can be—fast, powerful, and beautifully human.
0:32

Today, we’re proud to unveil a pioneering partnership with @Groq. Our Dialog voice model now runs on GroqCloud. This isn't just an upgrade; this is a leap forward that redefines what voice technology can be—fast, powerful, and beautifully human.

PlayAI

155,989 views • 1 year ago

Meet EVI 3, another step toward general voice intelligence. EVI 3 is a speech-language model that can understand and generate any human voice, not just a handful of speakers. With this broader voice intelligence comes greater expressiveness and a deeper understanding of tune, rhythm, timbre, and speaking style.
1:09

Meet EVI 3, another step toward general voice intelligence. EVI 3 is a speech-language model that can understand and generate any human voice, not just a handful of speakers. With this broader voice intelligence comes greater expressiveness and a deeper understanding of tune, rhythm, timbre, and speaking style.

Hume AI

832,639 views • 1 year ago

🚨NEW: CHATGPT LIVE VOICE MODEL NOW DELIVERS REAL-TIME CONVERSATIONAL TRANSLATION Built on a full-duplex architecture, GPT-Live is a new generation of voice models that can listen and speak simultaneously, enabling more natural human-AI interactions. The breakthrough unlocks fluid back-and-forth conversations, real-time translation, and a far more human sense of timing and context.
2:32

🚨NEW: CHATGPT LIVE VOICE MODEL NOW DELIVERS REAL-TIME CONVERSATIONAL TRANSLATION Built on a full-duplex architecture, GPT-Live is a new generation of voice models that can listen and speak simultaneously, enabling more natural human-AI interactions. The breakthrough unlocks fluid back-and-forth conversations, real-time translation, and a far more human sense of timing and context.

Coin Bureau

29,085 views • 26 days ago

This is wild. Hume AI just announced a new AI voice model. It’s like ChatGPT Advanced Voice Mode, Elevenlabs Voice Design, and Google NotebookLM in one. t can create a voice with a whole personality from a description or 5 second clip. And more. 8 wild examples:
1:25

This is wild. Hume AI just announced a new AI voice model. It’s like ChatGPT Advanced Voice Mode, Elevenlabs Voice Design, and Google NotebookLM in one. t can create a voice with a whole personality from a description or 5 second clip. And more. 8 wild examples:

Min Choi

533,611 views • 1 year ago

🚨BREAKING: A new open-source voice model just dropped and it’s better than ElevenLabs. This open-source voice AI is ultra-expressive, high quality, and fully free with no paywalls, no limits. Here’s how it works (with real examples):👇
0:26

🚨BREAKING: A new open-source voice model just dropped and it’s better than ElevenLabs. This open-source voice AI is ultra-expressive, high quality, and fully free with no paywalls, no limits. Here’s how it works (with real examples):👇

Sehaj Singh

814,129 views • 11 months ago

A music group called AlttA just dropped a new track featuring JAY-Z... kind of. They used an AI model to emulate his voice and the result is mind-blowing. Do you think it's only a matter of time until every artist has a precise vocal model at their fingertips?
0:40

A music group called AlttA just dropped a new track featuring JAY-Z... kind of. They used an AI model to emulate his voice and the result is mind-blowing. Do you think it's only a matter of time until every artist has a precise vocal model at their fingertips?

Eluna AI

178,707 views • 3 years ago

Miso One is an open-source voice model built for real-time conversations, responding in about 110 milliseconds while also cloning voices from short audio samples. Its speed makes Al voice chats feel smoother, faster, and much closer to natural human conversation
0:56

Miso One is an open-source voice model built for real-time conversations, responding in about 110 milliseconds while also cloning voices from short audio samples. Its speed makes Al voice chats feel smoother, faster, and much closer to natural human conversation

Hugging Models

15,754 views • 1 month ago

VoxCPM 2 just dropped by OpenBMB Only 2B-param open-source TTS (Text-to-Speech) model built for production-grade multilingual voice work. Apache-2.0 license, Can run on only 8GB VRAM. • Eliminates the "robotic" feel of traditional TTS, delivering prosody and emotional depth suitable for high-stakes professional environments like filmmaking, gaming, animation, and audiobooks. • 30-language multilingual: no language tag needed, just type in a supported language and generate directly. • Voice design: create a brand-new voice from a text description alone, like age, tone, pace, or emotion. No reference audio required. Describe the desired voice characteristics (gender, age, tone, emotion, pace …) in Control Instruction, and VoxCPM2 will craft a unique voice from your description alone. • Controllable cloning: clone from a short clip, then steer delivery style without losing the speaker’s core voice. • Ultimate cloning: use reference audio + transcript for continuation-style cloning that keeps the tiny vocal details. • 48kHz output: takes 16kHz reference audio and produces studio-quality speech without an external upsampler. • Real-time ready: around 0.3 RTF on RTX 4090, even lower with Nano-VLLM. • Commercial use: Apache-2.0 licensed. Developer-Friendly Infrastructure: - Native Torch Inference: Direct support for PyTorch-based workflows. - Training Flexibility: Supports both full-parameter and LoRA fine-tuning for specific domain adaptation. - Production Readiness: Compatible with voxcpm-nanovllm for large-scale, high-concurrency deployment.
1:58

VoxCPM 2 just dropped by OpenBMB Only 2B-param open-source TTS (Text-to-Speech) model built for production-grade multilingual voice work. Apache-2.0 license, Can run on only 8GB VRAM. • Eliminates the "robotic" feel of traditional TTS, delivering prosody and emotional depth suitable for high-stakes professional environments like filmmaking, gaming, animation, and audiobooks. • 30-language multilingual: no language tag needed, just type in a supported language and generate directly. • Voice design: create a brand-new voice from a text description alone, like age, tone, pace, or emotion. No reference audio required. Describe the desired voice characteristics (gender, age, tone, emotion, pace …) in Control Instruction, and VoxCPM2 will craft a unique voice from your description alone. • Controllable cloning: clone from a short clip, then steer delivery style without losing the speaker’s core voice. • Ultimate cloning: use reference audio + transcript for continuation-style cloning that keeps the tiny vocal details. • 48kHz output: takes 16kHz reference audio and produces studio-quality speech without an external upsampler. • Real-time ready: around 0.3 RTF on RTX 4090, even lower with Nano-VLLM. • Commercial use: Apache-2.0 licensed. Developer-Friendly Infrastructure: - Native Torch Inference: Direct support for PyTorch-based workflows. - Training Flexibility: Supports both full-parameter and LoRA fine-tuning for specific domain adaptation. - Production Readiness: Compatible with voxcpm-nanovllm for large-scale, high-concurrency deployment.

Rohan Paul

13,541 views • 3 months ago

ok this is insane.. i made an AI avatar SING a ballade with real emotion… and it actually looked like a scene from The Voice. this is the new AvatarX model and it’s scary good. tutorial + examples: ↓
0:22

ok this is insane.. i made an AI avatar SING a ballade with real emotion… and it actually looked like a scene from The Voice. this is the new AvatarX model and it’s scary good. tutorial + examples: ↓

The SamurAI

32,277 views • 11 months ago

This is insane. AI voice or real human? literally can’t tell the difference. OpenAudio S1 just dropped, this next-gen AI voice model, it nails emotions, accents, and dialogue like never before This is wild in audio, like Google Veo 3 in video. Donald J. Trump is in full fire mode on Elon
1:28

This is insane. AI voice or real human? literally can’t tell the difference. OpenAudio S1 just dropped, this next-gen AI voice model, it nails emotions, accents, and dialogue like never before This is wild in audio, like Google Veo 3 in video. Donald J. Trump is in full fire mode on Elon

Kaynat Kakar ✪

21,627 views • 1 year ago

I trained up an AI voice to sound exactly like me and then built a model that is super-interested in telemarketing calls but totally noncommittal, in order to drive them crazy... here's how it went!
4:05

I trained up an AI voice to sound exactly like me and then built a model that is super-interested in telemarketing calls but totally noncommittal, in order to drive them crazy... here's how it went!

Dave W Plummer

75,925 views • 1 year ago

How can a voice agent tell when you’re actually interrupting it? VAD is too sensitive—laughs, “mm-hmm,” or a sneeze shouldn’t stop the agent. We trained an audio model for adaptive interruption handling so agents can distinguish real interruptions from noise.
2:34

How can a voice agent tell when you’re actually interrupting it? VAD is too sensitive—laughs, “mm-hmm,” or a sneeze shouldn’t stop the agent. We trained an audio model for adaptive interruption handling so agents can distinguish real interruptions from noise.

LiveKit

43,832 views • 4 months ago

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.
1:21

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.

Rohan Paul

15,672 views • 4 months ago

It's over. Meta just announced MoCha, a new model that turns text or voice into super realistic talking characters. There is no way to tell anymore... 10 examples: (please unmute)
0:20

It's over. Meta just announced MoCha, a new model that turns text or voice into super realistic talking characters. There is no way to tell anymore... 10 examples: (please unmute)

Angry Tom

1,192,053 views • 1 year ago

Fish Audio has released S2.1 Pro, its latest production voice model that is built for real-time conversational AI. S2.1 Pro adds 👀 > Multi-speaker dialogue. > Voice cloning from short reference samples. > MCP plus agent-skill support for teams wiring speech into voice agents and long-form audio.
2:14

Fish Audio has released S2.1 Pro, its latest production voice model that is built for real-time conversational AI. S2.1 Pro adds 👀 > Multi-speaker dialogue. > Voice cloning from short reference samples. > MCP plus agent-skill support for teams wiring speech into voice agents and long-form audio.

🚨 AI News | TestingCatalog

333,682 views • 7 days ago