正在加载视频...

视频加载失败

返回首页

NEW: Kokoro 82M - APACHE 2.0 licensed, Text to Speech model, trained on < 100 hours of audio 🔥

reach_vb's profile picture

Vaibhav (VB) Srivastav

37,689 subscribers

330,034 次观看 • 1 年前 •via X (Twitter)

教育音乐科学技术
LIVE

Anya Rossi• Live Now

Private livecam show

11 条评论

Vaibhav (VB) Srivastav 的头像
Vaibhav (VB) Srivastav1 年前

Model checkpoint + voice pack here:

Vaibhav (VB) Srivastav 的头像
Vaibhav (VB) Srivastav1 年前

We also added it to TTS Arena, go play with it now:

The Information 的头像
The Information1 年前

The best new tech and business podcasts of 2024. Plus, podcasting's $400 million man.

mrfakename 的头像
mrfakename1 年前

Ranking #4 on the TTS Arena above some commercial models 🤯

Vaibhav (VB) Srivastav 的头像
Vaibhav (VB) Srivastav1 年前

🔥🔥🔥

Cavit Erginsoy 的头像
Cavit Erginsoy1 年前

did you say <100 hours?!

Vaibhav (VB) Srivastav 的头像
Vaibhav (VB) Srivastav1 年前

yes! synthetic data ftw!

Dyson Dunbar 的头像
Dyson Dunbar1 年前

its the first model that has convinced me to switch from xTTS because of the quality. It also helps that its about 10x faster as well on my 3090

Berat 的头像
Berat1 年前

<100 hours of audio with 20 epoch. grokking for the win

Vaibhav (VB) Srivastav 的头像
Vaibhav (VB) Srivastav1 年前

next step should be to fine-tune on Emilia

Pranjal Soni 的头像
Pranjal Soni1 年前

👀

相关视频

Kokoro is insane. 🤯 This AI is a groundbreaking TTS model with just 82M parameters. It outperforms larger models and generates minutes of speech in seconds. APACHE 2.0 licensed, trained on < 100 hours of audio is now available in ai-gradio pip install --upgrade "ai-gradio[kokoro]" import gradio as gr import ai_gradio gr.load(name='kokoro:kokoro-v0_19', src=ai_gradio.registry,).launch()
0:33

Kokoro is insane. 🤯 This AI is a groundbreaking TTS model with just 82M parameters. It outperforms larger models and generates minutes of speech in seconds. APACHE 2.0 licensed, trained on < 100 hours of audio is now available in ai-gradio pip install --upgrade "ai-gradio[kokoro]" import gradio as gr import ai_gradio gr.load(name='kokoro:kokoro-v0_19', src=ai_gradio.registry,).launch()

AK

114,000 次观看 • 1 年前

WOW! DeepMind *just* dropped Magenta Real-time - Apache 2.0 licensed 🔥 > 800M params transformer, trained on ~190K hours of instrumental stock music > adapts MusicLM for real-time generation via 2s audio chunks (conditioned on prior 10s context) > 48 KHz Stereo > MusicCoCa: New joint music-text embedding model, blending MuLan and CoCa approaches > 1.25s generation time for 2s audio on free-tier Colab TPUs > style embeddings (from text/audio prompts) allow real-time morphing of genres/instruments > on-device inference, personal fine-tuning - coming soon! > model weights on Hugging Face 🤗
1:02

WOW! DeepMind *just* dropped Magenta Real-time - Apache 2.0 licensed 🔥 > 800M params transformer, trained on ~190K hours of instrumental stock music > adapts MusicLM for real-time generation via 2s audio chunks (conditioned on prior 10s context) > 48 KHz Stereo > MusicCoCa: New joint music-text embedding model, blending MuLan and CoCa approaches > 1.25s generation time for 2s audio on free-tier Colab TPUs > style embeddings (from text/audio prompts) allow real-time morphing of genres/instruments > on-device inference, personal fine-tuning - coming soon! > model weights on Hugging Face 🤗

Vaibhav (VB) Srivastav

90,219 次观看 • 1 年前

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)
0:32

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)

Vaibhav (VB) Srivastav

82,126 次观看 • 1 年前

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 &gt; Outperforms ElevenLabs &gt; Trained on 500K hours of audio &gt; Llama 500M arch &gt; Emotionally aware speech &gt; Zero shot voice cloning &gt; Live on Hugging Face 🤗
0:44

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 &gt; Outperforms ElevenLabs &gt; Trained on 500K hours of audio &gt; Llama 500M arch &gt; Emotionally aware speech &gt; Zero shot voice cloning &gt; Live on Hugging Face 🤗

Vaibhav (VB) Srivastav

71,693 次观看 • 1 年前

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗
1:08

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗

Vaibhav (VB) Srivastav

66,963 次观看 • 1 年前

Introducing Indic-Parler TTS - Trained on 10K hours of data, 938M params, supports 20 Indic languages, emotional synthesis, apache 2.0 licensed! 🔥 A collaboration w/ AI4Bharat & Hugging Face - w/ fully customisable speech and voice personas! Try it out directly below or use the model weights as you want! 🇮🇳/acc
1:06

Introducing Indic-Parler TTS - Trained on 10K hours of data, 938M params, supports 20 Indic languages, emotional synthesis, apache 2.0 licensed! 🔥 A collaboration w/ AI4Bharat & Hugging Face - w/ fully customisable speech and voice personas! Try it out directly below or use the model weights as you want! 🇮🇳/acc

Vaibhav (VB) Srivastav

42,165 次观看 • 1 年前

Wait, that's JUST a 3B multimodal model understanding and generation model AND apache 2.0 licensed 🔥
0:34

Wait, that's JUST a 3B multimodal model understanding and generation model AND apache 2.0 licensed 🔥

Vaibhav (VB) Srivastav

43,777 次观看 • 1 年前

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️
1:23

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️

Rohan Paul

228,836 次观看 • 1 年前

End to End Speech models are on fire - LLAMA-OMNI 8B - Apache licensed! 🔥 > Speech Encoder - Whisper Large v3 > LLM backbone - Llama 3.1 8B Instruct > Speech Decoder - HuBERT (UnitY) > Simultaneously generate Speech + Text > Less than 250 ms latency > Trained in less than 3 days on 4x GPUs > Used 200K instruct pairs > Model checkpoints on the Hub 🤗 > Space incoming! GG! I'm here for this trend! 🐐
0:36

End to End Speech models are on fire - LLAMA-OMNI 8B - Apache licensed! 🔥 > Speech Encoder - Whisper Large v3 > LLM backbone - Llama 3.1 8B Instruct > Speech Decoder - HuBERT (UnitY) > Simultaneously generate Speech + Text > Less than 250 ms latency > Trained in less than 3 days on 4x GPUs > Used 200K instruct pairs > Model checkpoints on the Hub 🤗 > Space incoming! GG! I'm here for this trend! 🐐

Vaibhav (VB) Srivastav

47,921 次观看 • 1 年前

LETS GOO! Parler TTS 🔥 A fully open-source, Apache 2.0 licensed Text-to-speech model focused on providing maximum controllability. Through voice prompts, you can control the pitch, speed, gender, noise levels, emotion characteristics and more! > Trained on 10K hours of permissive data. > Offers control over the generations. > Training + Inference code released. > The processed dataset and tagging scripts were released for further research. > English only for now. Next, we're scaling the training to 50K hours and even better dataset processing! Want to help us out? DMs open! 🤗
0:39

LETS GOO! Parler TTS 🔥 A fully open-source, Apache 2.0 licensed Text-to-speech model focused on providing maximum controllability. Through voice prompts, you can control the pitch, speed, gender, noise levels, emotion characteristics and more! > Trained on 10K hours of permissive data. > Offers control over the generations. > Training + Inference code released. > The processed dataset and tagging scripts were released for further research. > English only for now. Next, we're scaling the training to 50K hours and even better dataset processing! Want to help us out? DMs open! 🤗

Vaibhav (VB) Srivastav

156,386 次观看 • 2 年前

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗
1:21

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

Vaibhav (VB) Srivastav

298,858 次观看 • 1 年前

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with &lt;300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice &amp; accent Try here -
0:40

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with &lt;300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice &amp; accent Try here -

PlayAI

392,571 次观看 • 2 年前

Pretty Insane - SoTA Text to Speech model capable of English AND Hindi - 3B Llama backbone - Apache 2.0 licensed 🔥 > Sub 80 ms latency > Supports both English, Hindi including code-mix > Runs in a free google colab too 🤯 Best part: They're actively working on other languages like Tamil, Telugu, Bengali, etc > Available on Hugging Face hub, powered by Transformers 💥
0:52

Pretty Insane - SoTA Text to Speech model capable of English AND Hindi - 3B Llama backbone - Apache 2.0 licensed 🔥 > Sub 80 ms latency > Supports both English, Hindi including code-mix > Runs in a free google colab too 🤯 Best part: They're actively working on other languages like Tamil, Telugu, Bengali, etc > Available on Hugging Face hub, powered by Transformers 💥

Vaibhav (VB) Srivastav

33,749 次观看 • 1 年前

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:
3:20

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

Santiago

66,397 次观看 • 10 个月前

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!
0:41

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!

Vaibhav (VB) Srivastav

93,512 次观看 • 1 年前

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.
0:45

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.

Vedant

15,714 次观看 • 1 个月前