Загрузка видео...

Не удалось загрузить видео

На главную

NEW: Kokoro 82M - APACHE 2.0 licensed, Text to Speech model, trained on < 100 hours of audio 🔥

reach_vb's profile picture

Vaibhav (VB) Srivastav

37,689 subscribers

330,034 просмотров • 1 год назад •via X (Twitter)

ОбразованиеМузыкаНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastav1 год назад

Model checkpoint + voice pack here:

Фото профиля Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastav1 год назад

We also added it to TTS Arena, go play with it now:

Фото профиля The Information
The Information1 год назад

The best new tech and business podcasts of 2024. Plus, podcasting's $400 million man.

Фото профиля mrfakename
mrfakename1 год назад

Ranking #4 on the TTS Arena above some commercial models 🤯

Фото профиля Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastav1 год назад

🔥🔥🔥

Фото профиля Cavit Erginsoy
Cavit Erginsoy1 год назад

did you say <100 hours?!

Фото профиля Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastav1 год назад

yes! synthetic data ftw!

Фото профиля Dyson Dunbar
Dyson Dunbar1 год назад

its the first model that has convinced me to switch from xTTS because of the quality. It also helps that its about 10x faster as well on my 3090

Фото профиля Berat
Berat1 год назад

<100 hours of audio with 20 epoch. grokking for the win

Фото профиля Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastav1 год назад

next step should be to fine-tune on Emilia

Фото профиля Pranjal Soni
Pranjal Soni1 год назад

👀

Похожие видео

Kokoro is insane. 🤯 This AI is a groundbreaking TTS model with just 82M parameters. It outperforms larger models and generates minutes of speech in seconds. APACHE 2.0 licensed, trained on < 100 hours of audio is now available in ai-gradio pip install --upgrade "ai-gradio[kokoro]" import gradio as gr import ai_gradio gr.load(name='kokoro:kokoro-v0_19', src=ai_gradio.registry,).launch()
0:33

Kokoro is insane. 🤯 This AI is a groundbreaking TTS model with just 82M parameters. It outperforms larger models and generates minutes of speech in seconds. APACHE 2.0 licensed, trained on < 100 hours of audio is now available in ai-gradio pip install --upgrade "ai-gradio[kokoro]" import gradio as gr import ai_gradio gr.load(name='kokoro:kokoro-v0_19', src=ai_gradio.registry,).launch()

AK

114,000 просмотров • 1 год назад

WOW! DeepMind *just* dropped Magenta Real-time - Apache 2.0 licensed 🔥 > 800M params transformer, trained on ~190K hours of instrumental stock music > adapts MusicLM for real-time generation via 2s audio chunks (conditioned on prior 10s context) > 48 KHz Stereo > MusicCoCa: New joint music-text embedding model, blending MuLan and CoCa approaches > 1.25s generation time for 2s audio on free-tier Colab TPUs > style embeddings (from text/audio prompts) allow real-time morphing of genres/instruments > on-device inference, personal fine-tuning - coming soon! > model weights on Hugging Face 🤗
1:02

WOW! DeepMind *just* dropped Magenta Real-time - Apache 2.0 licensed 🔥 > 800M params transformer, trained on ~190K hours of instrumental stock music > adapts MusicLM for real-time generation via 2s audio chunks (conditioned on prior 10s context) > 48 KHz Stereo > MusicCoCa: New joint music-text embedding model, blending MuLan and CoCa approaches > 1.25s generation time for 2s audio on free-tier Colab TPUs > style embeddings (from text/audio prompts) allow real-time morphing of genres/instruments > on-device inference, personal fine-tuning - coming soon! > model weights on Hugging Face 🤗

Vaibhav (VB) Srivastav

90,219 просмотров • 1 год назад

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)
0:32

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)

Vaibhav (VB) Srivastav

82,126 просмотров • 1 год назад

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 &gt; Outperforms ElevenLabs &gt; Trained on 500K hours of audio &gt; Llama 500M arch &gt; Emotionally aware speech &gt; Zero shot voice cloning &gt; Live on Hugging Face 🤗
0:44

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 &gt; Outperforms ElevenLabs &gt; Trained on 500K hours of audio &gt; Llama 500M arch &gt; Emotionally aware speech &gt; Zero shot voice cloning &gt; Live on Hugging Face 🤗

Vaibhav (VB) Srivastav

71,693 просмотров • 1 год назад

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗
1:08

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗

Vaibhav (VB) Srivastav

66,963 просмотров • 1 год назад

Introducing Indic-Parler TTS - Trained on 10K hours of data, 938M params, supports 20 Indic languages, emotional synthesis, apache 2.0 licensed! 🔥 A collaboration w/ AI4Bharat & Hugging Face - w/ fully customisable speech and voice personas! Try it out directly below or use the model weights as you want! 🇮🇳/acc
1:06

Introducing Indic-Parler TTS - Trained on 10K hours of data, 938M params, supports 20 Indic languages, emotional synthesis, apache 2.0 licensed! 🔥 A collaboration w/ AI4Bharat & Hugging Face - w/ fully customisable speech and voice personas! Try it out directly below or use the model weights as you want! 🇮🇳/acc

Vaibhav (VB) Srivastav

42,165 просмотров • 1 год назад

Wait, that's JUST a 3B multimodal model understanding and generation model AND apache 2.0 licensed 🔥
0:34

Wait, that's JUST a 3B multimodal model understanding and generation model AND apache 2.0 licensed 🔥

Vaibhav (VB) Srivastav

43,777 просмотров • 1 год назад

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️
1:23

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️

Rohan Paul

228,836 просмотров • 1 год назад

End to End Speech models are on fire - LLAMA-OMNI 8B - Apache licensed! 🔥 > Speech Encoder - Whisper Large v3 > LLM backbone - Llama 3.1 8B Instruct > Speech Decoder - HuBERT (UnitY) > Simultaneously generate Speech + Text > Less than 250 ms latency > Trained in less than 3 days on 4x GPUs > Used 200K instruct pairs > Model checkpoints on the Hub 🤗 > Space incoming! GG! I'm here for this trend! 🐐
0:36

End to End Speech models are on fire - LLAMA-OMNI 8B - Apache licensed! 🔥 > Speech Encoder - Whisper Large v3 > LLM backbone - Llama 3.1 8B Instruct > Speech Decoder - HuBERT (UnitY) > Simultaneously generate Speech + Text > Less than 250 ms latency > Trained in less than 3 days on 4x GPUs > Used 200K instruct pairs > Model checkpoints on the Hub 🤗 > Space incoming! GG! I'm here for this trend! 🐐

Vaibhav (VB) Srivastav

47,921 просмотров • 1 год назад

LETS GOO! Parler TTS 🔥 A fully open-source, Apache 2.0 licensed Text-to-speech model focused on providing maximum controllability. Through voice prompts, you can control the pitch, speed, gender, noise levels, emotion characteristics and more! > Trained on 10K hours of permissive data. > Offers control over the generations. > Training + Inference code released. > The processed dataset and tagging scripts were released for further research. > English only for now. Next, we're scaling the training to 50K hours and even better dataset processing! Want to help us out? DMs open! 🤗
0:39

LETS GOO! Parler TTS 🔥 A fully open-source, Apache 2.0 licensed Text-to-speech model focused on providing maximum controllability. Through voice prompts, you can control the pitch, speed, gender, noise levels, emotion characteristics and more! > Trained on 10K hours of permissive data. > Offers control over the generations. > Training + Inference code released. > The processed dataset and tagging scripts were released for further research. > English only for now. Next, we're scaling the training to 50K hours and even better dataset processing! Want to help us out? DMs open! 🤗

Vaibhav (VB) Srivastav

156,386 просмотров • 2 лет назад

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗
1:21

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

Vaibhav (VB) Srivastav

298,858 просмотров • 1 год назад

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with &lt;300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice &amp; accent Try here -
0:40

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with &lt;300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice &amp; accent Try here -

PlayAI

392,571 просмотров • 2 лет назад

Pretty Insane - SoTA Text to Speech model capable of English AND Hindi - 3B Llama backbone - Apache 2.0 licensed 🔥 > Sub 80 ms latency > Supports both English, Hindi including code-mix > Runs in a free google colab too 🤯 Best part: They're actively working on other languages like Tamil, Telugu, Bengali, etc > Available on Hugging Face hub, powered by Transformers 💥
0:52

Pretty Insane - SoTA Text to Speech model capable of English AND Hindi - 3B Llama backbone - Apache 2.0 licensed 🔥 > Sub 80 ms latency > Supports both English, Hindi including code-mix > Runs in a free google colab too 🤯 Best part: They're actively working on other languages like Tamil, Telugu, Bengali, etc > Available on Hugging Face hub, powered by Transformers 💥

Vaibhav (VB) Srivastav

33,749 просмотров • 1 год назад

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:
3:20

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

Santiago

66,397 просмотров • 10 месяцев назад

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!
0:41

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!

Vaibhav (VB) Srivastav

93,512 просмотров • 1 год назад

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.
0:45

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.

Vedant

15,714 просмотров • 1 месяц назад