Loading video...

Video Failed to Load

Go Home

Introducing Fish Speech 1.5 ๐ŸŽ‰ - Making state-of-the-art TTS accessible to everyone! Highlights: - #2 ranked on TTS-Arena (as "Anonymous Sparkle") - 1M hours of multilingual training data - 13 languages supported, including English, Chinese, Japanese & more - <150ms latency with high-quality instant voice cloning - Pretrained model...

101,606 views โ€ข 1 year ago โ€ขvia X (Twitter)

10 Comments

Fish Audio's profile picture
Fish Audio1 year ago

Try Fish Speech 1.5: ๐ŸŽฎ Playground: ๐Ÿ’ป Code: ๐ŸŽฏ Demo: ๐Ÿ† See our rank: Let's revolutionize voice tech together! ๐Ÿ 

Fish Audio's profile picture
Fish Audio1 year ago

Example: News

Fish Audio's profile picture
Fish Audio1 year ago

Example: Story Telling

Fish Audio's profile picture
Fish Audio1 year ago

Example: Cross-Lingual

Fish Audio's profile picture
Fish Audio1 year ago

A heartfelt thank you to our exceptional team for achieving this breakthrough in open-source TTS technology. Join us: We're seeking distinguished sales professionals and engineers. Contact [email protected] to shape the future of voice technology.

Alex Volkov (Thursd/AI)'s profile picture
Alex Volkov (Thursd/AI)1 year ago

Congrats on this release! Will cover tomorrow on @thursdai_pod !

Stas Bichenko's profile picture
Stas Bichenko1 year ago

Self-hosted - only for non-commercial use, right?

Yossi Dahan's profile picture
Yossi Dahan1 year ago

Where can I find a list of supported languages?

GenAI Alien's profile picture
GenAI Alien1 year ago

Letโ€™s go! Fire ๐Ÿ”ฅ! Love it!

Roman's profile picture
Roman1 year ago

Looks great! May you add other languages (like french) in the online demo and samples? Thanks !

Related Videos

VoxCPM 2 just dropped by OpenBMB Only 2B-param open-source TTS (Text-to-Speech) model built for production-grade multilingual voice work. Apache-2.0 license, Can run on only 8GB VRAM. โ€ข Eliminates the "robotic" feel of traditional TTS, delivering prosody and emotional depth suitable for high-stakes professional environments like filmmaking, gaming, animation, and audiobooks. โ€ข 30-language multilingual: no language tag needed, just type in a supported language and generate directly. โ€ข Voice design: create a brand-new voice from a text description alone, like age, tone, pace, or emotion. No reference audio required. Describe the desired voice characteristics (gender, age, tone, emotion, pace โ€ฆ) in Control Instruction, and VoxCPM2 will craft a unique voice from your description alone. โ€ข Controllable cloning: clone from a short clip, then steer delivery style without losing the speakerโ€™s core voice. โ€ข Ultimate cloning: use reference audio + transcript for continuation-style cloning that keeps the tiny vocal details. โ€ข 48kHz output: takes 16kHz reference audio and produces studio-quality speech without an external upsampler. โ€ข Real-time ready: around 0.3 RTF on RTX 4090, even lower with Nano-VLLM. โ€ข Commercial use: Apache-2.0 licensed. Developer-Friendly Infrastructure: - Native Torch Inference: Direct support for PyTorch-based workflows. - Training Flexibility: Supports both full-parameter and LoRA fine-tuning for specific domain adaptation. - Production Readiness: Compatible with voxcpm-nanovllm for large-scale, high-concurrency deployment.

Rohan Paul

13,541 views โ€ข 3 months ago

๐Ÿšจ JUST IN: MICROSOFT just open sourced a VOICE AI THAT TRANSCRIBES 60 MINUTES OF AUDIO in a single pass. 100% FREE. It knows who spoke. It knows when they spoke. It knows exactly what they said. All in one shot. No chunking. No context loss. It's called VibeVoice. Not a transcription tool. Not a basic speech to text wrapper. A frontier voice AI family with ASR, TTS, and real time streaming. All open source. All free. Here's what it actually does ๐Ÿ‘‡ VibeVoice ASR - Speech Recognition: โ†’ Processes 60 minutes of continuous audio in a single pass โ†’ Never slices audio into chunks so global context is never lost โ†’ Identifies WHO spoke, WHEN they spoke and WHAT they said simultaneously โ†’ Supports customized hotwords for domain specific accuracy โ†’ Works in 50+ languages natively โ†’ Already adopted by Hugging Face Transformers library โ†’ Already being built on by the open source community BY PEOPLE WHO HAD NO IDEA THIS LEVEL OF ACCURACY WAS ALREADY FREE. VibeVoice TTS - Text to Speech: โ†’ Generates up to 90 minutes of speech in a single pass โ†’ Supports up to 4 distinct speakers in one conversation โ†’ Natural turn taking and speaker consistency throughout โ†’ Expressive speech that captures emotional nuances โ†’ Supports English, Chinese and multiple other languages VibeVoice Realtime - Streaming TTS: โ†’ Only 300 millisecond first audible latency โ†’ Streams text input in real time โ†’ 0.5B parameters so it actually deploys anywhere โ†’ Robust long form generation up to 10 minutes โ†’ Lightweight enough for production use today The core innovation nobody is talking about: Most voice AI models slice long audio into short chunks. Every time they slice, they lose context. Speaker tracking breaks. Semantic coherence breaks. Accuracy drops. VibeVoice uses continuous speech tokenizers running at an ultra low frame rate of 7.5 Hz. This preserves audio fidelity while dramatically boosting computational efficiency. The entire 60 minutes stays in context. Nothing gets lost. Nobody gets misidentified. The numbers: โ†’ VibeVoice ASR 7B - available now on Hugging Face โ†’ VibeVoice Realtime 0.5B - try it on Colab right now โ†’ 50+ supported languages โ†’ 11 distinct English voice styles โ†’ 9 multilingual speaker voices โ†’ Already integrated into Hugging Face Transformers โ†’ Finetuning code now available The wildest part? A voice powered input method called Vibing just built itself on top of VibeVoice ASR. Available on macOS and Windows right now. The open source community is already shipping products on top of this. 100% Open Source. Free to use. Free to fine tune. Free to build on. ๐Ÿ”– Save this before your competitors find it first. ๐Ÿ‘‡

Kanika

220,656 views โ€ข 3 months ago

What if your voice AI could interrupt you the moment it figured out your question - sometimes even before you finished asking it? Last week, I sat down with Neil, CEO of Gradium and co-founder of Kyutai , to talk about the future of speech-to-speech models and why he believes today's cascaded voice systems will soon look "archaic and brittle." Some highlights from our conversation: ๐ŸŽฏ How Kyutai built Moshiโ€”a full duplex conversational AI with "negative latency"โ€”in 6 months with just 4-6 people (while big tech teams had 10-20x the resources) ๐Ÿง  Why speech-to-speech models lose intelligence compared to their text counterparts (and what's being done about it) ๐Ÿ“ฑ Pocket TTS: The first voice cloning model that runs on your phone's CPUโ€”not GPU, CPU ๐Ÿค– Why robotics and spatial audio represent the next frontier (hint: current voice systems completely break in these environments) ๐Ÿ‘ถ The efficiency gap: Babies learn to speak fluently from <5,000 hours of audio. Current models train on millions of hours. We're doing something wrong. My favorite vision from Neil? The first truly contrarian AI that interrupts you mid-sentence to tell you why you're wrong. Not just more natural conversationโ€”but actually useful for testing ideas and playing devil's advocate. Full episode and detailed blog post linked in the comments ๐Ÿ‘‡ What's your take - will speech-to-speech replace cascaded systems, or will modularity keep cascaded architectures dominant even as naturalness improves?

Brooke Hopkins

13,000 views โ€ข 4 months ago

QVAC SDK 0.14.0 is live. This release makes the on-device stack faster on mobile, ships the developer-agent path, and takes local text-to-speech to 31 languages. Main highlights: - OpenCode and OpenClaw. The first official OpenCode plugin, plus a maintained OpenClaw compatibility path, both built on managed mode and qvac serve. Point a coding agent at a local model with far less setup and far fewer surprises. - Brain-computer interface transcription, on the SDK. Take recorded neural signal data and decode it into text, fully on-device, no cloud. Stream it in chunks through a simple API. In 0.14 it runs GPU-accelerated on iOS. - Text to Speech in 31 languages with our Supertonic3 upgrade. VOICE AND SPEECH - Supertonic3 multilingual TTS, 5 languages to 31. - Chatterbox and Supertonic now run on the Android GPU, with lower memory use (especially on iOS), quantized s3gen Chatterbox support, and a fix for Chatterbox occasionally emitting random speech. - Whisper transcription now runs on the iOS GPU. Parakeet runs on the Android GPU, with steadier real-time streaming. VISION AND OCR - VLM multi-tile batching: high-resolution Pan and Scan images are encoded in one pass instead of tile by tile, for faster vision throughput. - OCR on ggml (EasyOCR and DocTR) reaches full speed parity with the onnx path, across Metal, OpenCL, and Vulkan. PLATFORM AND RELIABILITY - Dynamic compute backends on Linux: one build picks the right backend at runtime, and opens the door to ROCm and CUDA support without per-backend builds. - Thinking tokens are kept out of the model context, so reasoning no longer fills the KV cache. SDK 0.14.0 is now leaner and faster to start. Letโ€™s build.

QVAC

23,973,950 views โ€ข 16 days ago

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

kwindla

40,319 views โ€ข 1 month ago

Google just proved that bigger isn't always better. Their 308M parameter model is outperforming models 2x its size. Google just released ๐—˜๐—บ๐—ฏ๐—ฒ๐—ฑ๐—ฑ๐—ถ๐—ป๐—ด๐—š๐—ฒ๐—บ๐—บ๐—ฎ, and it's proving that lightweight embedding models can punch way above their weight class. At just 308M parameters (578MB), it's the new state-of-the-art for models under 500M parameters across MTEB multilingual, English, and code benchmarks. But the really impressive part is that it ranks 8th overall on MTEB(Multilingual, v2) - that's ๐Ÿญ๐Ÿณ ๐—ฝ๐—น๐—ฎ๐—ฐ๐—ฒ๐˜€ above the second-best sub-500M model, and it's delivering performance ๐—ฐ๐—ผ๐—บ๐—ฝ๐—ฎ๐—ฟ๐—ฎ๐—ฏ๐—น๐—ฒ ๐˜๐—ผ ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น๐˜€ ๐—ป๐—ฒ๐—ฎ๐—ฟ๐—น๐˜† ๐—ฑ๐—ผ๐˜‚๐—ฏ๐—น๐—ฒ ๐—ถ๐˜๐˜€ ๐˜€๐—ถ๐˜‡๐—ฒ. There are three key parts of their training recipe that sets it apart: ๐Ÿญ. ๐—˜๐—ป๐—ฐ๐—ผ๐—ฑ๐—ฒ๐—ฟ-๐——๐—ฒ๐—ฐ๐—ผ๐—ฑ๐—ฒ๐—ฟ ๐—œ๐—ป๐—ถ๐˜๐—ถ๐—ฎ๐—น๐—ถ๐˜‡๐—ฎ๐˜๐—ถ๐—ผ๐—ป Instead of starting from a decoder-only Gemma 3 model, they first adapted it to encoder-decoder, then used just the encoder. By basing EmbeddingGemma off an LLM that already has world and language understanding, it gives it a stronger starting point. ๐Ÿฎ. ๐—ง๐—ต๐—ฟ๐—ฒ๐—ฒ-๐—Ÿ๐—ผ๐˜€๐˜€ ๐—ง๐—ฟ๐—ฎ๐—ถ๐—ป๐—ถ๐—ป๐—ด They combine three different loss functions, instead of just having one: โ€ข Contrastive loss (NCE) with in-batch negatives and hardness weighting โ€ข Spread-out regularization to ensure embeddings utilize the full space (for quantization and ANN retrieval) โ€ข Embedding matching distillation from Gemini Embedding - not just learning from relevance scores, but directly aligning the embedding space with the teacher model ๐Ÿฏ. ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ฆ๐—ผ๐˜‚๐—ฝ๐—ถ๐—ป๐—ด Rather than just averaging checkpoints from the same training run, they use optimization techniques to find multiple specialized training mixtures. Each mixture creates an "expert" model in different domains, and averaging all their parameters creates a final model that's actually better than individual models. Extras: โ€ข Matryoshka embeddings supporting 768, 512, 256, and 128 dimensions โ€ข Quantization-aware training - maintains quality even at int4 precision โ€ข 100+ languages from Gemma 3 pretraining โ€ข Exceptional performance on low-resource languages (check their XTREME-UP results) Is it the absolute best embedding model? No - Gemini Embedding still leads overall. But that's not really the point. EmbeddingGemma proves you can achieve state-of-the-art performance in a small package that's actually deployable on-device, in low-latency applications, and in resource-constrained environments. This makes good embeddings accessible for use cases that I'm seeing more and more: offline applications, privacy-sensitive deployments, and high-throughput scenarios where inference cost actually matters. Full paper: Shoutout to the EmbeddingGemma team at Google DeepMind for this awesome open source work ๐Ÿ’™ and to Daniel Williams for helping me with this video! ๐Ÿซถ

Victoria Slocum

21,592 views โ€ข 7 months ago

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.๐ŸŽจ โœจ New in 2.1: ๐Ÿ”นAdvanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. ๐Ÿ”นPrecise Chinese and English Text Rendering with seamless imageโ€“text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. ๐Ÿ”นRich Styles and High Aesthetic: Capable of generating images in various stylesโ€”including photorealistic portraits, comics, and vinyl figuresโ€”it delivers outstanding visual appeal and artistic quality. ๐Ÿ”นHigh-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideasโ€”like posters with slogans or multi-panel comicsโ€”into visuals faster than ever. Weโ€™re just getting started. Stay tuned for our native multimodal image generation model coming soon. ๐ŸŒWebsite: ๐Ÿ”—Github: ๐Ÿค—Hugging Face: โœจHugging Face Demo:

Tencent Hy

89,257 views โ€ข 10 months ago

In Q1 2024, The Graph Network saw substantial growth in queries, reaching an all-time high of over 1.5 billion, up 65% from just shy of 1 billion in Q4โ€™23. This growth accompanies updates recently shipped by core devs, enabling developers to easily take advantage of low cost, reliable and permissionless decentralized data โšก Here are the top 8 key takeaways from the Q1 2024 Participant Update โฌ‡๏ธ 1๏ธโƒฃ The Graph Network experienced significant growth, serving 1.5+ billion queriesโ€”a 65% quarter-over-quarter increase. Additionally, the number of subgraphs published on the network grew by over 30%, exceeding 1,900 subgraphs. 2๏ธโƒฃ The Sunray Phase of the Sunrise of Decentralized Data concluded with the launch of key developer centric features including a free query plan, payment by credit card or GRT with access to 5๏ธโƒฃ0๏ธโƒฃ+ chains on the network. 3๏ธโƒฃ The Graph added support for Bitcoin data by introducing a Bitcoin explorer module, a composable BRC-20 Substreams Module and a BRC-20 Substreams module, facilitating the integration of data into subgraphs for GraphQL queries. 4๏ธโƒฃ Members of The Graphโ€™s leadership teams traveled to Asia and participated in community events, engaged with the local web3 communities and strengthened The Graph ecosystemโ€™s relationships with key thought leaders in the region. 5๏ธโƒฃ Looking ahead, Sunbeam (Phase 2 of the Sunrise of Decentralized Data) concludes on June 12th. This milestone marks the end of the subgraph upgrade window in which hosted service subgraphs must upgrade to the network. 6๏ธโƒฃ Firehose is now integrated with Go Ethereum. The 44% of Ethereum devs that make use of GETH can now get natively deeper insights, more precise event ordering, pattern matching, simpler and a more reliable integration, lower latency and super fast reprocessing times. 7๏ธโƒฃ The Graph is saving the blobs! Developers now have two ways to access blob data (EIP-4844) - through either the consensus layer or the execution layer. 8๏ธโƒฃ The Graph ecosystem can expect many exciting announcements coming in the near future around new data services live on the network and served by an ecosystem of Indexers earning GRT. Core Devs (Edge & Node, StreamingFast โซ, Semiotic AI, The Guild, Messari, GraphOps | graphops.eth, Pinax, and ) will continue to execute on the New Era Roadmap by integrating SQL query capability and a SQL data service, File Hosting Service, new micropayment system Scalar TAP, AI data services and more. The Graph's Q1 2024 Participant Update establishes a clear watermark for web3 and decentralized data. The Graphโ€™s expansion across many chains as well as progress on new developer experience enhancing features and new data services is expanding whatโ€™s possible with decentralized data in web3. Accessing more diverse data across more blockchains has never been easier for dapp developers and data consumers. The best part: The Graph has many more big announcements to come throughout 2024 ๐ŸŒŸ Donโ€™t miss any of the details. Watch the full update featuring Eva Beylin, Tegan Kline, 0xMaxTang, Etienne Brunet, and caro f here:

The Graph

27,400 views โ€ข 2 years ago