Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

NEW: Kokoro 82M - APACHE 2.0 licensed, Text to Speech model, trained on < 100 hours of audio 🔥

reach_vb's profile picture

Vaibhav (VB) Srivastav

37,689 subscribers

330,034 görüntüleme • 1 yıl önce •via X (Twitter)

EğitimMüzikBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

11 Yorum

Vaibhav (VB) Srivastav profil fotoğrafı
Vaibhav (VB) Srivastav1 yıl önce

Model checkpoint + voice pack here:

Vaibhav (VB) Srivastav profil fotoğrafı
Vaibhav (VB) Srivastav1 yıl önce

We also added it to TTS Arena, go play with it now:

The Information profil fotoğrafı
The Information1 yıl önce

The best new tech and business podcasts of 2024. Plus, podcasting's $400 million man.

mrfakename profil fotoğrafı
mrfakename1 yıl önce

Ranking #4 on the TTS Arena above some commercial models 🤯

Vaibhav (VB) Srivastav profil fotoğrafı
Vaibhav (VB) Srivastav1 yıl önce

🔥🔥🔥

Cavit Erginsoy profil fotoğrafı
Cavit Erginsoy1 yıl önce

did you say <100 hours?!

Vaibhav (VB) Srivastav profil fotoğrafı
Vaibhav (VB) Srivastav1 yıl önce

yes! synthetic data ftw!

Dyson Dunbar profil fotoğrafı
Dyson Dunbar1 yıl önce

its the first model that has convinced me to switch from xTTS because of the quality. It also helps that its about 10x faster as well on my 3090

Berat profil fotoğrafı
Berat1 yıl önce

<100 hours of audio with 20 epoch. grokking for the win

Vaibhav (VB) Srivastav profil fotoğrafı
Vaibhav (VB) Srivastav1 yıl önce

next step should be to fine-tune on Emilia

Pranjal Soni profil fotoğrafı
Pranjal Soni1 yıl önce

👀

Benzer Videolar

Kokoro is insane. 🤯 This AI is a groundbreaking TTS model with just 82M parameters. It outperforms larger models and generates minutes of speech in seconds. APACHE 2.0 licensed, trained on < 100 hours of audio is now available in ai-gradio pip install --upgrade "ai-gradio[kokoro]" import gradio as gr import ai_gradio gr.load(name='kokoro:kokoro-v0_19', src=ai_gradio.registry,).launch()
0:33

Kokoro is insane. 🤯 This AI is a groundbreaking TTS model with just 82M parameters. It outperforms larger models and generates minutes of speech in seconds. APACHE 2.0 licensed, trained on < 100 hours of audio is now available in ai-gradio pip install --upgrade "ai-gradio[kokoro]" import gradio as gr import ai_gradio gr.load(name='kokoro:kokoro-v0_19', src=ai_gradio.registry,).launch()

AK

114,000 görüntüleme • 1 yıl önce

WOW! DeepMind *just* dropped Magenta Real-time - Apache 2.0 licensed 🔥 > 800M params transformer, trained on ~190K hours of instrumental stock music > adapts MusicLM for real-time generation via 2s audio chunks (conditioned on prior 10s context) > 48 KHz Stereo > MusicCoCa: New joint music-text embedding model, blending MuLan and CoCa approaches > 1.25s generation time for 2s audio on free-tier Colab TPUs > style embeddings (from text/audio prompts) allow real-time morphing of genres/instruments > on-device inference, personal fine-tuning - coming soon! > model weights on Hugging Face 🤗
1:02

WOW! DeepMind *just* dropped Magenta Real-time - Apache 2.0 licensed 🔥 > 800M params transformer, trained on ~190K hours of instrumental stock music > adapts MusicLM for real-time generation via 2s audio chunks (conditioned on prior 10s context) > 48 KHz Stereo > MusicCoCa: New joint music-text embedding model, blending MuLan and CoCa approaches > 1.25s generation time for 2s audio on free-tier Colab TPUs > style embeddings (from text/audio prompts) allow real-time morphing of genres/instruments > on-device inference, personal fine-tuning - coming soon! > model weights on Hugging Face 🤗

Vaibhav (VB) Srivastav

90,219 görüntüleme • 1 yıl önce

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)
0:32

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)

Vaibhav (VB) Srivastav

82,126 görüntüleme • 1 yıl önce

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 &gt; Outperforms ElevenLabs &gt; Trained on 500K hours of audio &gt; Llama 500M arch &gt; Emotionally aware speech &gt; Zero shot voice cloning &gt; Live on Hugging Face 🤗
0:44

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 &gt; Outperforms ElevenLabs &gt; Trained on 500K hours of audio &gt; Llama 500M arch &gt; Emotionally aware speech &gt; Zero shot voice cloning &gt; Live on Hugging Face 🤗

Vaibhav (VB) Srivastav

71,693 görüntüleme • 1 yıl önce

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗
1:08

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/ 200ms TTFA > Models on the Hub & Finetuning code on its way! 🚀 What an amazing time to be alive 🤗

Vaibhav (VB) Srivastav

66,963 görüntüleme • 1 yıl önce

Introducing Indic-Parler TTS - Trained on 10K hours of data, 938M params, supports 20 Indic languages, emotional synthesis, apache 2.0 licensed! 🔥 A collaboration w/ AI4Bharat & Hugging Face - w/ fully customisable speech and voice personas! Try it out directly below or use the model weights as you want! 🇮🇳/acc
1:06

Introducing Indic-Parler TTS - Trained on 10K hours of data, 938M params, supports 20 Indic languages, emotional synthesis, apache 2.0 licensed! 🔥 A collaboration w/ AI4Bharat & Hugging Face - w/ fully customisable speech and voice personas! Try it out directly below or use the model weights as you want! 🇮🇳/acc

Vaibhav (VB) Srivastav

42,165 görüntüleme • 1 yıl önce

Wait, that's JUST a 3B multimodal model understanding and generation model AND apache 2.0 licensed 🔥
0:34

Wait, that's JUST a 3B multimodal model understanding and generation model AND apache 2.0 licensed 🔥

Vaibhav (VB) Srivastav

43,777 görüntüleme • 1 yıl önce

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️
1:23

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️

Rohan Paul

228,836 görüntüleme • 1 yıl önce

End to End Speech models are on fire - LLAMA-OMNI 8B - Apache licensed! 🔥 > Speech Encoder - Whisper Large v3 > LLM backbone - Llama 3.1 8B Instruct > Speech Decoder - HuBERT (UnitY) > Simultaneously generate Speech + Text > Less than 250 ms latency > Trained in less than 3 days on 4x GPUs > Used 200K instruct pairs > Model checkpoints on the Hub 🤗 > Space incoming! GG! I'm here for this trend! 🐐
0:36

End to End Speech models are on fire - LLAMA-OMNI 8B - Apache licensed! 🔥 > Speech Encoder - Whisper Large v3 > LLM backbone - Llama 3.1 8B Instruct > Speech Decoder - HuBERT (UnitY) > Simultaneously generate Speech + Text > Less than 250 ms latency > Trained in less than 3 days on 4x GPUs > Used 200K instruct pairs > Model checkpoints on the Hub 🤗 > Space incoming! GG! I'm here for this trend! 🐐

Vaibhav (VB) Srivastav

47,921 görüntüleme • 1 yıl önce

LETS GOO! Parler TTS 🔥 A fully open-source, Apache 2.0 licensed Text-to-speech model focused on providing maximum controllability. Through voice prompts, you can control the pitch, speed, gender, noise levels, emotion characteristics and more! > Trained on 10K hours of permissive data. > Offers control over the generations. > Training + Inference code released. > The processed dataset and tagging scripts were released for further research. > English only for now. Next, we're scaling the training to 50K hours and even better dataset processing! Want to help us out? DMs open! 🤗
0:39

LETS GOO! Parler TTS 🔥 A fully open-source, Apache 2.0 licensed Text-to-speech model focused on providing maximum controllability. Through voice prompts, you can control the pitch, speed, gender, noise levels, emotion characteristics and more! > Trained on 10K hours of permissive data. > Offers control over the generations. > Training + Inference code released. > The processed dataset and tagging scripts were released for further research. > English only for now. Next, we're scaling the training to 50K hours and even better dataset processing! Want to help us out? DMs open! 🤗

Vaibhav (VB) Srivastav

156,386 görüntüleme • 2 yıl önce

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗
1:21

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

Vaibhav (VB) Srivastav

298,858 görüntüleme • 1 yıl önce

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with &lt;300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice &amp; accent Try here -
0:40

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with &lt;300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice &amp; accent Try here -

PlayAI

392,571 görüntüleme • 2 yıl önce

Pretty Insane - SoTA Text to Speech model capable of English AND Hindi - 3B Llama backbone - Apache 2.0 licensed 🔥 > Sub 80 ms latency > Supports both English, Hindi including code-mix > Runs in a free google colab too 🤯 Best part: They're actively working on other languages like Tamil, Telugu, Bengali, etc > Available on Hugging Face hub, powered by Transformers 💥
0:52

Pretty Insane - SoTA Text to Speech model capable of English AND Hindi - 3B Llama backbone - Apache 2.0 licensed 🔥 > Sub 80 ms latency > Supports both English, Hindi including code-mix > Runs in a free google colab too 🤯 Best part: They're actively working on other languages like Tamil, Telugu, Bengali, etc > Available on Hugging Face hub, powered by Transformers 💥

Vaibhav (VB) Srivastav

33,749 görüntüleme • 1 yıl önce

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:
3:20

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

Santiago

66,397 görüntüleme • 10 ay önce

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!
0:41

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!

Vaibhav (VB) Srivastav

93,512 görüntüleme • 1 yıl önce

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.
0:45

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.

Vedant

15,714 görüntüleme • 1 ay önce