Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Wow! New Speech to Speech model - Fish Agent v0.1 3B by Fish Audio 🔥 > Trained on 700K hours of multilingual audio > Continue-pretrained version of Qwen-2.5-3B-Instruct for 200B audio & text tokens > Zero-shot voice cloning > Text + audio input/ Audio output > Ultra-fast inference w/...

reach_vb's profile picture

Vaibhav (VB) Srivastav

58,407 subscribers

67,003 görüntüleme • 1 yıl önce •via X (Twitter)

SanatEğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

10 Yorum

Vaibhav (VB) Srivastav profil fotoğrafı
Vaibhav (VB) Srivastav1 yıl önce

Check out the model here:

Vaibhav (VB) Srivastav profil fotoğrafı
Vaibhav (VB) Srivastav1 yıl önce

They also put together a pretty dope space to try out the model:

Juan Pablo Gallego profil fotoğrafı
Juan Pablo Gallego1 yıl önce

@huggingface @FishAudio It was clearly trained on TTS voice. Sounds very robotic

William Payne profil fotoğrafı
William Payne1 yıl önce

@FishAudio How does it compare to Mochi?

Ishtiaq Rahman profil fotoğrafı
Ishtiaq Rahman1 yıl önce

@FishAudio Unfortunately, 3+ seconds for non-RAG questions is slow as a mule. OpenAI's speech-to-speech is much, much faster. Naah, this ain't it.

Houdini profil fotoğrafı
Houdini1 yıl önce

@FishAudio Youre always early on these bangers. Nice work.

Houdini profil fotoğrafı
Houdini1 yıl önce

@FishAudio @cocktailpeanut

bl4nk profil fotoğrafı
bl4nk1 yıl önce

@FishAudio Pls drop google colab code

Sadi Moodi profil fotoğrafı
Sadi Moodi1 yıl önce

@FishAudio no published code to use, so its useless

Sahil 🇮🇳/acc profil fotoğrafı
Sahil 🇮🇳/acc1 yıl önce

@FishAudio why do these guys don't do hindi?

Benzer Videolar

LETS GOOO! FishSpeech v1.5 - multilingual, zero-shot instant voice cloning, low-latency, open text to speech model 🔥 > Only 500M params > Trained on 1 MILLION hours of audio > Supports 13 languages > Low-latency ( Open model - checkpoints on the hub 🤗 > Best part: Ranks #2 on TTS Arena (preliminary results) Kudos to Fish Audio - truly leading the way for open text to speech research! ⚡
2:26

LETS GOOO! FishSpeech v1.5 - multilingual, zero-shot instant voice cloning, low-latency, open text to speech model 🔥 > Only 500M params > Trained on 1 MILLION hours of audio > Supports 13 languages > Low-latency ( Open model - checkpoints on the hub 🤗 > Best part: Ranks #2 on TTS Arena (preliminary results) Kudos to Fish Audio - truly leading the way for open text to speech research! ⚡

Vaibhav (VB) Srivastav

49,150 görüntüleme • 1 yıl önce

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗
1:21

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

Vaibhav (VB) Srivastav

299,066 görüntüleme • 1 yıl önce

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️
1:23

The new open-source Text to Speech model: Fish Speech 1.4 is brilliant! Trained on a massive 700K hours of multilingual speech data in 8 languages - Instant voice cloning 🗣️ - Ultra-low latency ⚡ - Compact model (~1GB weights) 🏋️‍♂️

Rohan Paul

228,905 görüntüleme • 2 yıl önce

NEW: Kokoro 82M - APACHE 2.0 licensed, Text to Speech model, trained on < 100 hours of audio 🔥
1:54

NEW: Kokoro 82M - APACHE 2.0 licensed, Text to Speech model, trained on < 100 hours of audio 🔥

Vaibhav (VB) Srivastav

330,034 görüntüleme • 1 yıl önce

Smol TTS keeps getting better! Introducing OuteTTS v0.2 - 500M parameters, multilingual with voice cloning! 🔥 > Multilingual - English, Chinese, Korean & Japanese > Cross platform inference w/ llama.cpp > Zero-shot voice cloning > Trained on 5 Billion audio tokens > Qwen 2.5 0.5B LLM backbone > Trained via HF GPU grants Model weights on the hub, you can even run this on a Raspberry Pi! Go run, inference now! 🐐
0:50

Smol TTS keeps getting better! Introducing OuteTTS v0.2 - 500M parameters, multilingual with voice cloning! 🔥 > Multilingual - English, Chinese, Korean & Japanese > Cross platform inference w/ llama.cpp > Zero-shot voice cloning > Trained on 5 Billion audio tokens > Qwen 2.5 0.5B LLM backbone > Trained via HF GPU grants Model weights on the hub, you can even run this on a Raspberry Pi! Go run, inference now! 🐐

Vaibhav (VB) Srivastav

44,691 görüntüleme • 1 yıl önce

NVIDIA Releases Audex (Nemotron-Labs-Audex-30B-A3B): A Unified Audio-Text LLM That Preserves the Text Intelligence of Its Backbone Most unified audio models pay a text tax. Add audio output, and reasoning benchmarks drop — even when the only new output is speech. NVIDIA just released one that doesn't. Audex (Nemotron-Labs-Audex-30B-A3B) is a 30B MoE with 3B active parameters, built on the text-only Nemotron-Cascade-2-30B-A3B backbone. Audio inputs are projected into the text embedding space. Text tokens and quantized audio tokens are then generated the same way, inside one MoE decoder — no thinker–talker split, no stacked cascade. Here's what's actually interesting: → One model, audio in and out: understanding, ASR, translation, TTS, text-to-audio, speech-to-speech → Text holds vs its own backbone: IMO AnswerBench 81.1 vs 79.3, MMLU-Redux 86.4 vs 86.3 → Beats text-only Qwen3.5-35B-A3B on several tasks: LiveCodeBench v6 85.3 vs 74.6, IFBench 77.8 vs 70.2 → The usual tax, for contrast: Qwen3-Omni-30B-A3B-Thinking drops to 60.4 on HMMT vs 71.4 for its text backbone → 6.82 WER on OpenASR, ahead of Step-Audio-R1.1-33B (7.91) and Qwen3-Omni-Thinking (8.00) → Two codecs: X-Codec2 for speech (50 tok/s, FSQ, 65,536 codebook), X-Codec for general audio (200 tok/s, 4 flattened RVQ layers) — the only strong open model generating general audio beyond speech Full analysis: Paper: Model weights: NVIDIA AI NVIDIA Wei Ping
0:27

NVIDIA Releases Audex (Nemotron-Labs-Audex-30B-A3B): A Unified Audio-Text LLM That Preserves the Text Intelligence of Its Backbone Most unified audio models pay a text tax. Add audio output, and reasoning benchmarks drop — even when the only new output is speech. NVIDIA just released one that doesn't. Audex (Nemotron-Labs-Audex-30B-A3B) is a 30B MoE with 3B active parameters, built on the text-only Nemotron-Cascade-2-30B-A3B backbone. Audio inputs are projected into the text embedding space. Text tokens and quantized audio tokens are then generated the same way, inside one MoE decoder — no thinker–talker split, no stacked cascade. Here's what's actually interesting: → One model, audio in and out: understanding, ASR, translation, TTS, text-to-audio, speech-to-speech → Text holds vs its own backbone: IMO AnswerBench 81.1 vs 79.3, MMLU-Redux 86.4 vs 86.3 → Beats text-only Qwen3.5-35B-A3B on several tasks: LiveCodeBench v6 85.3 vs 74.6, IFBench 77.8 vs 70.2 → The usual tax, for contrast: Qwen3-Omni-30B-A3B-Thinking drops to 60.4 on HMMT vs 71.4 for its text backbone → 6.82 WER on OpenASR, ahead of Step-Audio-R1.1-33B (7.91) and Qwen3-Omni-Thinking (8.00) → Two codecs: X-Codec2 for speech (50 tok/s, FSQ, 65,536 codebook), X-Codec for general audio (200 tok/s, 4 flattened RVQ layers) — the only strong open model generating general audio beyond speech Full analysis: Paper: Model weights: NVIDIA AI NVIDIA Wei Ping

Marktechpost AI

28,301 görüntüleme • 2 ay önce

PaLM + AudioLM = AudioPaLM ! We start from PaLM pretrained on text and extend its vocab w/ audio tokens. This model can then be finetuned on a mix of any (speech, text) task e.g. ASR, TTS, MT and speech2speech translation in one's voice! 🧵1/4
2:01

PaLM + AudioLM = AudioPaLM ! We start from PaLM pretrained on text and extend its vocab w/ audio tokens. This model can then be finetuned on a mix of any (speech, text) task e.g. ASR, TTS, MT and speech2speech translation in one's voice! 🧵1/4

Neil Zeghidour

41,263 görüntüleme • 3 yıl önce

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.
0:45

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.

Vedant

15,953 görüntüleme • 4 ay önce

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 > Outperforms ElevenLabs > Trained on 500K hours of audio > Llama 500M arch > Emotionally aware speech > Zero shot voice cloning > Live on Hugging Face 🤗
0:44

Pretty fucking wild - Chatterbox TTS by Resemble AI - Zero shot voice cloning, Apache 2.0 licensed 🤯 > Outperforms ElevenLabs > Trained on 500K hours of audio > Llama 500M arch > Emotionally aware speech > Zero shot voice cloning > Live on Hugging Face 🤗

Vaibhav (VB) Srivastav

71,770 görüntüleme • 1 yıl önce

Introducing Dialog 1.0 - Ultra-emotional AI Text-To-Speech model Outperforms Elevenlabs on expressiveness and quality 3 to 1 <1% error rate Supports 30+ languages Best in class voice cloning Low latency: 303ms TTFA (Time to First Audio) Experience it for yourself on Read more below⬇️
0:51

Introducing Dialog 1.0 - Ultra-emotional AI Text-To-Speech model Outperforms Elevenlabs on expressiveness and quality 3 to 1 <1% error rate Supports 30+ languages Best in class voice cloning Low latency: 303ms TTFA (Time to First Audio) Experience it for yourself on Read more below⬇️

PlayAI

196,182 görüntüleme • 1 yıl önce

NEW: Higgs Audio V2 from BosonAI open, unified TTS model w/ voice cloning, beats GPT 4o mini tts and ElevenLabs v2 🔥 > Trained on 10M hours (speech, music, events) > Built on top of Llama 3.2 3B > Works real-time and on edge > Beats GPT-4o-mini-tts, ElevenLabs v2 in prosody & emotion Multi-speaker dialog > Zero-shot voice cloning 🤩 > Available on Hugging Face Kudos to folks at Boson AI for releasing such a brilliant work and all the details around the model! 🤗
2:10

NEW: Higgs Audio V2 from BosonAI open, unified TTS model w/ voice cloning, beats GPT 4o mini tts and ElevenLabs v2 🔥 > Trained on 10M hours (speech, music, events) > Built on top of Llama 3.2 3B > Works real-time and on edge > Beats GPT-4o-mini-tts, ElevenLabs v2 in prosody & emotion Multi-speaker dialog > Zero-shot voice cloning 🤩 > Available on Hugging Face Kudos to folks at Boson AI for releasing such a brilliant work and all the details around the model! 🤗

Vaibhav (VB) Srivastav

79,645 görüntüleme • 1 yıl önce

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)
4:07

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,553 görüntüleme • 1 yıl önce

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)
0:32

Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥 > WhisperSpeech X Llama 3.1 8B > Trained on 50K hours of speech (7 languages) > Continually trained on 45hrs 10x A1000s > MLS -> WhisperVQ tokens -> Llama 3.1 > Instruction tuned on 1.89M samples > 70% speech, 20% transcription, 10% text > Apache 2.0 licensed ⚡ Architecture: > WhisperSpeech/ VQ for Semantic Tokens > Llama 3.1 8B Instruct for Text backbone > Early fusion (Chameleon) I'm super bullish on Homebrew to Menlo and early fusion, audio and text, multimodal models! (P.S. Play with the demo on Hugging Face)

Vaibhav (VB) Srivastav

82,170 görüntüleme • 1 yıl önce

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!
0:41

Kyutai released their Streaming Text to Speech model, ~2B param model, ultra low latency (220ms), CC-BY-4.0 license 🔥 Trained on 2.5 Million Hours of audio, it can serve up to 32 users w/ less than 350ms latency on a SINGLE L40 🤯 Incredible release by kyutai folks, go check out their hugging face page now!

Vaibhav (VB) Srivastav

93,586 görüntüleme • 1 yıl önce

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:
3:20

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

Santiago

66,420 görüntüleme • 1 yıl önce

ElevenLabs has officially LOST to Open-Source ResembleAI allows you to clone ANY voice without verification using on 5-10 seconds of audio, and dominates on paralinguistic tags for human-like expressions. Most "fast" text-to-speech models sound robotic. Most "quality" TTS models are slow. None incorporate authentication at a foundational level. Resemble AI solved all three. Chatterbox Turbo delivers: 🟢<150ms time-to-first-sound 🟢State-of-the-art quality that beats larger proprietary models 🟢Natural, programmable expressions 🟢Zero-shot voice cloning with just 5 seconds of audio 🟢PerTh watermarking for authenticated and verifiable audio 🟢Open source – full transparency, no black boxes Try it on HuggingFace:
0:23

ElevenLabs has officially LOST to Open-Source ResembleAI allows you to clone ANY voice without verification using on 5-10 seconds of audio, and dominates on paralinguistic tags for human-like expressions. Most "fast" text-to-speech models sound robotic. Most "quality" TTS models are slow. None incorporate authentication at a foundational level. Resemble AI solved all three. Chatterbox Turbo delivers: 🟢<150ms time-to-first-sound 🟢State-of-the-art quality that beats larger proprietary models 🟢Natural, programmable expressions 🟢Zero-shot voice cloning with just 5 seconds of audio 🟢PerTh watermarking for authenticated and verifiable audio 🟢Open source – full transparency, no black boxes Try it on HuggingFace:

AI Breakfast

208,873 görüntüleme • 9 ay önce

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️
4:29

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️

Prince Canuma

164,026 görüntüleme • 7 ay önce

Real-time AI conversations are here! PlayHT, one of the best text-to-speech models I’ve used, now has a latency of less than 300ms. Checkout how fast it outputs audio 🤯 Also, my experience cloning my own voice and links to try it for free are below.
0:30

Real-time AI conversations are here! PlayHT, one of the best text-to-speech models I’ve used, now has a latency of less than 300ms. Checkout how fast it outputs audio 🤯 Also, my experience cloning my own voice and links to try it for free are below.

Alvaro Cintas

335,389 görüntüleme • 2 yıl önce

Excited to introduce Fish Speech 1.4 - now open-source and more powerful than ever! 🎉 Our mission is to make cutting-edge voice tech accessible to everyone. What's new: - Trained on 700k hours of multilingual data (up from 200k) - Now supports 8 languages: English, Chinese, German, Japanese, French, Spanish, Korean, and Arabic - Fully open-source, empowering developers and researchers worldwide Key features: - Lightning-fast TTS with ultra-low latency - Instant voice cloning - Self-host or use our cloud service - Simple, flat-rate pricing Try it out: - Playground: - GitHub: - HuggingFace Model: - Demo: - Product Hunt: We can't wait to see what you'll create with Fish Audio. Happy voice building! 🎧🐠
1:23

Excited to introduce Fish Speech 1.4 - now open-source and more powerful than ever! 🎉 Our mission is to make cutting-edge voice tech accessible to everyone. What's new: - Trained on 700k hours of multilingual data (up from 200k) - Now supports 8 languages: English, Chinese, German, Japanese, French, Spanish, Korean, and Arabic - Fully open-source, empowering developers and researchers worldwide Key features: - Lightning-fast TTS with ultra-low latency - Instant voice cloning - Self-host or use our cloud service - Simple, flat-rate pricing Try it out: - Playground: - GitHub: - HuggingFace Model: - Demo: - Product Hunt: We can't wait to see what you'll create with Fish Audio. Happy voice building! 🎧🐠

Fish Audio

149,977 görüntüleme • 2 yıl önce