Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Today, we’re releasing Octave: the first LLM built for text-to-speech. 🎨Design any voice with a prompt 🎬 Give acting instructions to control emotion and delivery (sarcasm, whispering, etc.) 🛠️Produce long-form content on our Creator Studio Unlike traditional TTS that just “reads” words aloud, Octave understands how meaning affects delivery... show more

Hume AI

23,229 subscribers

394,020 views • 1 year ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

12 Comments

Hume1 year ago

🎨Voice Design Create any AI voice with a simple prompt. From "Southern ASMR meditation coach" to "film noir detective" — Octave instantly generates the voices you need for your content. Octave beats ElevenLabs Voice Design in a robust eval. Find out more👇

Hume1 year ago

🎬Acting Instructions Octave is the first TTS system that can take natural language instructions to change emotional delivery and speaking style. Give directions like "sound sarcastic" or "whisper fearfully." For the first time, creators have total control.

Hume1 year ago

🤔Context-Aware Expression Trained on 1000x more language than traditional TTS, Octave understands your script like a human actor, delivering realistic emotions, sarcasm, pace, word emphasis, and more. This allows it to understand plot twists, emotional cues, character traits, and how to combine them—it reads love letters tenderly and sports announcements energetically.

Hume1 year ago

🛠️Tools for Creators and Developers Use our creator studio to edit and generate long-form content precisely with acting instructions. For developers, Octave is accessible through Python and TypeScript SDKs that handle authentication and provide typed interfaces for reliable integration.

Hume1 year ago

Octave’s first-of-its-kind voice intelligence speaks for itself 😉 In a blind study, Octave outperformed ElevenLabs Voice Design: 🔊71.6% preferred Octave’s audio quality 🗣️51.7% found Octave more natural 🎯57.7% said Octave better matched voice descriptions

Hume1 year ago

Create with Octave today: And the best part? Even with superior capabilities, Octave is cheaper than alternatives. Our blog shares more about Octave, our blind study, and what’s next:

Hume1 year ago

Join our Discord to show off your projects, share product feedback, get technical support, or just hang with our empathic community. We are so excited to see what you create!

TurboScribe1 year ago

What REAL customers say about TurboScribe's unlimited AI transcription 👇 🇺🇸 "Not only does it transcribe with amazing accuracy, it also filters out a ton of the unnecessary noise associated with pauses in audio. Keep up the great work!" - Kevin

Moses Oh1 year ago

Wait … are all the voices in this video AI generated?

Hume1 year ago

R u serious Moses

Rohan Paul1 year ago

This is superb.. love it.. Characters finally speak in context without heavy manual adjustment.

Hume1 year ago

thank you!! show us what you create :)

Related Videos

What if voice AI could understand context, intent, and emotion — not just words? 🎙️ Meet Seed Speech 2.0. A new speech AI stack designed for natural, expressive conversations — with major upgrades in both text-to-speech (TTS) and speech recognition (ASR). Built for developers and creators who want natural conversational speech. What’s inside: - Natural and expressive speech generation - Prompt-controlled emotion and tone - High-accuracy multilingual recognition - Strong contextual reasoning Built for: 🎧 Voice content creation 🤖 AI assistants ☎️ Customer service 🎬 Dubbing & subtitling 📊 Audio-video analysis Voice AI built for expression, accuracy, and understanding. Try it now → Read more about Seed Speech here → #SeedSpeech #VoiceAI #SpeechAI #AI

What if voice AI could understand context, intent, and emotion — not just words? 🎙️ Meet Seed Speech 2.0. A new speech AI stack designed for natural, expressive conversations — with major upgrades in both text-to-speech (TTS) and speech recognition (ASR). Built for developers and creators who want natural conversational speech. What’s inside: - Natural and expressive speech generation - Prompt-controlled emotion and tone - High-accuracy multilingual recognition - Strong contextual reasoning Built for: 🎧 Voice content creation 🤖 AI assistants ☎️ Customer service 🎬 Dubbing & subtitling 📊 Audio-video analysis Voice AI built for expression, accuracy, and understanding. Try it now → Read more about Seed Speech here → #SeedSpeech #VoiceAI #SpeechAI #AI

BytePlus

19,740 views • 5 months ago

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,764 views • 1 year ago

Today we’re excited to introduce Bland TTS, the first voice AI to cross the uncanny valley. Several months ago, our team solved one-shot style transfer of human speech. That means, from a single, brief MP3, you can clone any voice or remix another clone’s style (tone, cadence, or pronunciations, etc.). With Bland TTS: • Creatives can turn text into realistic sound effects or AI voice tracks with precise control over style and emotion. • Developers can plug in our TTS API to build voice apps fully customized to their liking. • Enterprises can launch AI-driven support lines that sound so natural — customers will save you as a contact (try talking to it on our site). 🧵Check out the thread below or visit our website to get started for free

Today we’re excited to introduce Bland TTS, the first voice AI to cross the uncanny valley. Several months ago, our team solved one-shot style transfer of human speech. That means, from a single, brief MP3, you can clone any voice or remix another clone’s style (tone, cadence, or pronunciations, etc.). With Bland TTS: • Creatives can turn text into realistic sound effects or AI voice tracks with precise control over style and emotion. • Developers can plug in our TTS API to build voice apps fully customized to their liking. • Enterprises can launch AI-driven support lines that sound so natural — customers will save you as a contact (try talking to it on our site). 🧵Check out the thread below or visit our website to get started for free

Bland

45,968,162 views • 1 year ago

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Vaibhav Sisinty

24,535,026 views • 4 months ago

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

HOLY FUCK! Zyphra just dropped Zonos - Apache 2.0 licensed, Multilingual, Text to Speech model with INSTANT voice cloning! 🔥 > Zero-shot TTS with Voice Cloning: Input text and a 10-30 second speaker sample to generate high-quality text-to-speech output > Audio Prefix Inputs: Enhance speaker matching by adding an audio prefix to the text, enabling behaviors like whispering that are hard to achieve with voice cloning alone > Multilingual Support: Supports English, Japanese, Chinese, French, and German > Audio Quality & Emotion Control: Fine-tune speaking rate, pitch, frequency, audio quality, and emotions (e.g., happiness, anger, sadness, fear) > Fast Performance: Runs at ~2x real-time speed on an RTX 4090 > Available on the Hugging Face Hub 🤗

Vaibhav (VB) Srivastav

299,016 views • 1 year ago

Introducing FlowSpeech: The World’s First TTS to Turn Written Text into Natural Speech Every other TTS service on the market today simply reads the text word-for-word. It’s like giving a presentation by just reading your PowerPoint slides out loud. It doesn’t work. You need to present it naturally, in your own words. And that’s the principle behind FlowSpeech. FlowSpeech is the world’s first TTS that can transform formal written text into conversational, spoken language. A TTS that actually sounds human. That might sound a bit abstract, so let your ears be the judge.

Introducing FlowSpeech: The World’s First TTS to Turn Written Text into Natural Speech Every other TTS service on the market today simply reads the text word-for-word. It’s like giving a presentation by just reading your PowerPoint slides out loud. It doesn’t work. You need to present it naturally, in your own words. And that’s the principle behind FlowSpeech. FlowSpeech is the world’s first TTS that can transform formal written text into conversational, spoken language. A TTS that actually sounds human. That might sound a bit abstract, so let your ears be the judge.

ListenHub

57,440 views • 1 year ago

🚀 Meet the new Qwen3-TTS lineup: VoiceDesign & VoiceClone! Create, control, and clone voices—faster and more expressive than ever. ⚡ VoiceDesign-VD-Flash • Fully controllable speech via free-form text instructions — tone, rhythm, emotion, persona • No preset voices. Design your own unique vocal identity • Outperforms GPT-4o-mini-tts & Gemini-2.5-pro on role-play benchmarks ⚡ VoiceClone-VC-Flash • Clone any voice from just 3 seconds of audio • Generate speech in 10 languages (CN / EN / JP / ES + more) • 15% lower WER vs. ElevenLabs & GPT-4o-Audio in multilingual tests • Context-aware cadence for more natural delivery 🎙️ Try it now • Qwen Chat: • Blog:

🚀 Meet the new Qwen3-TTS lineup: VoiceDesign & VoiceClone! Create, control, and clone voices—faster and more expressive than ever. ⚡ VoiceDesign-VD-Flash • Fully controllable speech via free-form text instructions — tone, rhythm, emotion, persona • No preset voices. Design your own unique vocal identity • Outperforms GPT-4o-mini-tts & Gemini-2.5-pro on role-play benchmarks ⚡ VoiceClone-VC-Flash • Clone any voice from just 3 seconds of audio • Generate speech in 10 languages (CN / EN / JP / ES + more) • 15% lower WER vs. ElevenLabs & GPT-4o-Audio in multilingual tests • Context-aware cadence for more natural delivery 🎙️ Try it now • Qwen Chat: • Blog:

Qwen

180,928 views • 8 months ago

VoxCPM 2 just dropped by OpenBMB Only 2B-param open-source TTS (Text-to-Speech) model built for production-grade multilingual voice work. Apache-2.0 license, Can run on only 8GB VRAM. • Eliminates the "robotic" feel of traditional TTS, delivering prosody and emotional depth suitable for high-stakes professional environments like filmmaking, gaming, animation, and audiobooks. • 30-language multilingual: no language tag needed, just type in a supported language and generate directly. • Voice design: create a brand-new voice from a text description alone, like age, tone, pace, or emotion. No reference audio required. Describe the desired voice characteristics (gender, age, tone, emotion, pace …) in Control Instruction, and VoxCPM2 will craft a unique voice from your description alone. • Controllable cloning: clone from a short clip, then steer delivery style without losing the speaker’s core voice. • Ultimate cloning: use reference audio + transcript for continuation-style cloning that keeps the tiny vocal details. • 48kHz output: takes 16kHz reference audio and produces studio-quality speech without an external upsampler. • Real-time ready: around 0.3 RTF on RTX 4090, even lower with Nano-VLLM. • Commercial use: Apache-2.0 licensed. Developer-Friendly Infrastructure: - Native Torch Inference: Direct support for PyTorch-based workflows. - Training Flexibility: Supports both full-parameter and LoRA fine-tuning for specific domain adaptation. - Production Readiness: Compatible with voxcpm-nanovllm for large-scale, high-concurrency deployment.

Rohan Paul

13,541 views • 4 months ago

Today, we’re excited to introduce Miso One, the most emotive voice model in the world. Miso One is an 8-billion-parameter text-to-speech model for highly expressive speech generation. It emotes like a human and responds faster than a human, with just 110 milliseconds of latency. We’ve open-sourced the model weights, with API access coming soon. Hear how Miso One sounds in the thread below.

Today, we’re excited to introduce Miso One, the most emotive voice model in the world. Miso One is an 8-billion-parameter text-to-speech model for highly expressive speech generation. It emotes like a human and responds faster than a human, with just 110 milliseconds of latency. We’ve open-sourced the model weights, with API access coming soon. Hear how Miso One sounds in the thread below.

Aoden Teo

5,224,593 views • 2 months ago

Introducing Soniox TTS v2, our most powerful text-to-speech model yet. Soniox TTS v2 brings extraordinary voice quality, expressive control through audio tags, exceptional precision, high-fidelity voice cloning, more than 60 languages, natural language mixing, and low-latency streaming together in one model. Built from the ground up with a new model architecture, audio codec, and highly optimized inference engine, Soniox TTS v2 delivers frontier TTS at just $0.70 per generated hour. Available globally today as tts-rt-v2. Try it out: Turn the sound on and hear what’s possible. 🔊

Introducing Soniox TTS v2, our most powerful text-to-speech model yet. Soniox TTS v2 brings extraordinary voice quality, expressive control through audio tags, exceptional precision, high-fidelity voice cloning, more than 60 languages, natural language mixing, and low-latency streaming together in one model. Built from the ground up with a new model architecture, audio codec, and highly optimized inference engine, Soniox TTS v2 delivers frontier TTS at just $0.70 per generated hour. Available globally today as tts-rt-v2. Try it out: Turn the sound on and hear what’s possible. 🔊

Soniox

644,246 views • 15 days ago

people always ask me how they can create long, consistent AI videos with audio. well creatify aurora solves a big part of that and there are some new models coming soon too for more action + audio. I created his voice with dia tts which i think great at capturing emotional speeches. then I cloned his voice using minimax voice clone and use it on minimax tts 2.6. here is the my full conversation with claude for images and text to speech inputs:

people always ask me how they can create long, consistent AI videos with audio. well creatify aurora solves a big part of that and there are some new models coming soon too for more action + audio. I created his voice with dia tts which i think great at capturing emotional speeches. then I cloned his voice using minimax voice clone and use it on minimax tts 2.6. here is the my full conversation with claude for images and text to speech inputs:

gokaygokay

38,341 views • 8 months ago

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Stripe

18,535 views • 4 months ago

Text-to-speech is moving way too fast. Just a few days ago, I tweeted about PersonaPlex-7B, NVIDIA's new open source TTS ( And today, Qwen just open-sourced Qwen3-TTS 🤯 It’s a revolutionary text-to-speech model built for control. Not just about generating speech, but about shaping how it sounds directly from language. You can guide the pace, the tone, and the expressiveness straight from text, without touching audio graphs or hand-tuning parameters. That’s the real shift! What makes Qwen3-TTS stand out is how practical it already is: → voice cloning from just a few seconds of audio → voice creation without any reference sample → support for 10 languages out of the box → end-to-end latency down to ~97ms → works in both streaming and non-streaming setups The models come in two sizes (0.6B and 1.7B), so you can trade off quality and hardware cost depending on your setup. You can work with curated voices, designed voices, or cloned ones, and it integrates cleanly with vLLM for production use. It also ships as a simple Python package you can pip install. If you’re building real-time voice systems, this removes a lot of friction! 100% free and open source. I put the repo in the 🧵↓

Text-to-speech is moving way too fast. Just a few days ago, I tweeted about PersonaPlex-7B, NVIDIA's new open source TTS ( And today, Qwen just open-sourced Qwen3-TTS 🤯 It’s a revolutionary text-to-speech model built for control. Not just about generating speech, but about shaping how it sounds directly from language. You can guide the pace, the tone, and the expressiveness straight from text, without touching audio graphs or hand-tuning parameters. That’s the real shift! What makes Qwen3-TTS stand out is how practical it already is: → voice cloning from just a few seconds of audio → voice creation without any reference sample → support for 10 languages out of the box → end-to-end latency down to ~97ms → works in both streaming and non-streaming setups The models come in two sizes (0.6B and 1.7B), so you can trade off quality and hardware cost depending on your setup. You can work with curated voices, designed voices, or cloned ones, and it integrates cleanly with vLLM for production use. It also ships as a simple Python package you can pip install. If you’re building real-time voice systems, this removes a lot of friction! 100% free and open source. I put the repo in the 🧵↓

Charly Wargnier

59,144 views • 7 months ago

Today we’re releasing Chroma 1.0 → the world first open-source, end-to-end, real-time speech-to-speech model → with personalized voice cloning Trained by FlashLabs. Deployed on FlashAI👉 An open research-grade alternative to the OpenAI Realtime model. Voice Test dubbing Elon Musk and Lex Fridman: 🔥What’s real (evals and benchmarks attached): ⚡ 2× faster than real-time ) 📘 SGLang Cookbook: 📄 Paper + benchmarks: 🤗 Models: 💻 Inference code: 🔁 RT if you believe real-time AI should be open 💬 Reply if you’re building on Conversational Voice AI products

Today we’re releasing Chroma 1.0 → the world first open-source, end-to-end, real-time speech-to-speech model → with personalized voice cloning Trained by FlashLabs. Deployed on FlashAI👉 An open research-grade alternative to the OpenAI Realtime model. Voice Test dubbing Elon Musk and Lex Fridman: 🔥What’s real (evals and benchmarks attached): ⚡ 2× faster than real-time ) 📘 SGLang Cookbook: 📄 Paper + benchmarks: 🤗 Models: 💻 Inference code: 🔁 RT if you believe real-time AI should be open 💬 Reply if you’re building on Conversational Voice AI products

FlashLabs

1,248,841 views • 7 months ago

Brain signals and LLM embeddings converge for predicting every spoken or heard word. Beautiful research from Google AI They compared human brain activity during real conversations with internal embeddings from a speech-to-text LLM. Measured electrode signals in speech and language-related brain regions and matched them to the model’s word-level features. 🤖Key Highlights → Brain activity aligns linearly with LLM embeddings for real-life spoken conversations. → Sequence of comprehension: first speech sounds, then word meaning. → Sequence of production: planned meaning, then articulation, then hearing one’s own voice. → Consistent predictive coding (pre-onset anticipation, post-onset surprise) mirrors LLM next-word prediction. → Lower-tier auditory regions still show partial sensitivity to semantic information. 🤖 Model-Brain Alignment They observed a clear sequence: during comprehension, auditory cortex (superior temporal gyrus) showed strong correlation with speech embeddings, then language embeddings aligned with Broca’s area. During production, Broca’s area correlated with language embeddings before articulation, followed by motor cortex signals matching speech embeddings. This suggests that next-word prediction and higher-level meaning representation in the model parallel the brain’s approach. ⚙ So the study revealed a shared computational principle of predicting words in context. Even though the Transformer-based LLM processes words in parallel layers, the human brain processes them serially yet mirrors similar statistical regularities. This supports a “soft hierarchy” where both lower-level acoustic processing and higher-level semantic processing partially overlap in the brain.

Brain signals and LLM embeddings converge for predicting every spoken or heard word. Beautiful research from Google AI They compared human brain activity during real conversations with internal embeddings from a speech-to-text LLM. Measured electrode signals in speech and language-related brain regions and matched them to the model’s word-level features. 🤖Key Highlights → Brain activity aligns linearly with LLM embeddings for real-life spoken conversations. → Sequence of comprehension: first speech sounds, then word meaning. → Sequence of production: planned meaning, then articulation, then hearing one’s own voice. → Consistent predictive coding (pre-onset anticipation, post-onset surprise) mirrors LLM next-word prediction. → Lower-tier auditory regions still show partial sensitivity to semantic information. 🤖 Model-Brain Alignment They observed a clear sequence: during comprehension, auditory cortex (superior temporal gyrus) showed strong correlation with speech embeddings, then language embeddings aligned with Broca’s area. During production, Broca’s area correlated with language embeddings before articulation, followed by motor cortex signals matching speech embeddings. This suggests that next-word prediction and higher-level meaning representation in the model parallel the brain’s approach. ⚙ So the study revealed a shared computational principle of predicting words in context. Even though the Transformer-based LLM processes words in parallel layers, the human brain processes them serially yet mirrors similar statistical regularities. This supports a “soft hierarchy” where both lower-level acoustic processing and higher-level semantic processing partially overlap in the brain.

Rohan Paul

15,213 views • 1 year ago

This one says “Ya’ll hear that? That’s the sound of the saxophones getting louder for Chud. And let this serve as a reminder that ya’ll have freedom of speech but not freedom of f*cking consequences. So watch your f*cking mouth.” These people want to control what you’re allowed to say based on what offends them. They want to label any words they don’t like as “hate speech” but even “hate speech” is protected under the First Amendment, just like any other speech. As long as it’s not a direct threat or incitement to violence, no one has the legal right to physically attack you simply because they find your words offensive. Why is this so hard for them to understand? They say the most hateful things about Whites on a daily basis but only they can have a “no-no” word.

This one says “Ya’ll hear that? That’s the sound of the saxophones getting louder for Chud. And let this serve as a reminder that ya’ll have freedom of speech but not freedom of fcking consequences. So watch your fcking mouth.” These people want to control what you’re allowed to say based on what offends them. They want to label any words they don’t like as “hate speech” but even “hate speech” is protected under the First Amendment, just like any other speech. As long as it’s not a direct threat or incitement to violence, no one has the legal right to physically attack you simply because they find your words offensive. Why is this so hard for them to understand? They say the most hateful things about Whites on a daily basis but only they can have a “no-no” word.

Make Sense of it Marie

12,626 views • 3 months ago

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps: babbling starts turning into words ~50k steps: it reads anything you type (WER under 1%) ~200k steps: the voice stops sounding synthetic On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power. Some things we'd love to see people try: - Train it in your own language (a few hundred hours of speech gets you surprisingly far). - Add new features to Pocket TTS (Emotion tags? Make it sing?). - Beat us at our own game: make it faster and smaller. Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it! Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own. Get started:

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps: babbling starts turning into words ~50k steps: it reads anything you type (WER under 1%) ~200k steps: the voice stops sounding synthetic On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power. Some things we'd love to see people try: - Train it in your own language (a few hundred hours of speech gets you surprisingly far). - Add new features to Pocket TTS (Emotion tags? Make it sing?). - Beat us at our own game: make it faster and smaller. Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it! Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own. Get started:

kyutai

91,724 views • 1 day ago

This might be the best agent I've seen yet. After raising $220M, H just introduced an agent that can execute any task from a prompt. Their "Runner H" can basically turn instructions into action with human-like precision. Features: ▸ Navigates web interfaces with pixel-level precision. ▸ Interprets pixels and text to understand screens and elements. ▸ Automates workflows for web testing, onboarding, and e-commerce. ▸ Adapts automatically to UI changes. ▸ Achieves a 67% success rate on WebVoyager, outperforming competitors. Architecture: ▸ Powered by a 2B-paramezer LLM for function calling and coding. ▸ Includes a 3B-parameter VLM for understanding graphical and text elements.

This might be the best agent I've seen yet. After raising $220M, H just introduced an agent that can execute any task from a prompt. Their "Runner H" can basically turn instructions into action with human-like precision. Features: ▸ Navigates web interfaces with pixel-level precision. ▸ Interprets pixels and text to understand screens and elements. ▸ Automates workflows for web testing, onboarding, and e-commerce. ▸ Adapts automatically to UI changes. ▸ Achieves a 67% success rate on WebVoyager, outperforming competitors. Architecture: ▸ Powered by a 2B-paramezer LLM for function calling and coding. ▸ Includes a 3B-parameter VLM for understanding graphical and text elements.

Lior Alexander

106,814 views • 1 year ago

WE ARE LAUNCHING!! 🚀 We’ve been working nonstop the past few weeks... Today, we’re finally launching Genviral ✨ A short-form content automation tool that makes it insanely easy to generate and post videos on social media. 🧠 Drop your website – we scan it, understand your product, and generate viral hooks automatically 👤 Create your own AI influencers or use existing ones 🎬 Use our AI Studio to modify them with just a prompt: outfits, styles, scenes, you name it 📊 Generate carousels with 1 click 📱 Post and schedule content to TikTok (Instagram coming soon) across multiple accounts We’ve put everything we’ve got into making Genviral as simple and powerful as possible. Over the next few weeks, we’ll go deep into short-form content: analyzing trends, reverse-engineering what works, and shipping updates fast. Hope you guys like it!

WE ARE LAUNCHING!! 🚀 We’ve been working nonstop the past few weeks... Today, we’re finally launching Genviral ✨ A short-form content automation tool that makes it insanely easy to generate and post videos on social media. 🧠 Drop your website – we scan it, understand your product, and generate viral hooks automatically 👤 Create your own AI influencers or use existing ones 🎬 Use our AI Studio to modify them with just a prompt: outfits, styles, scenes, you name it 📊 Generate carousels with 1 click 📱 Post and schedule content to TikTok (Instagram coming soon) across multiple accounts We’ve put everything we’ve got into making Genviral as simple and powerful as possible. Over the next few weeks, we’ll go deep into short-form content: analyzing trends, reverse-engineering what works, and shipping updates fast. Hope you guys like it!

Fekri

31,170 views • 1 year ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 views • 3 years ago