Loading video...

Video Failed to Load

Go Home

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps:...

106,234 views • 1 month ago •via X (Twitter)

34 Comments

pulsatingGenius's profile picture
pulsatingGenius1 month ago

Thanks for releasing the training code it matches what i was doing. only issue was mine kept blabbering and eos token prediction had issues

0xShugo's profile picture
0xShugo1 month ago

PocketTTS --- the fastest model in the audio.cpp Arena!

Josh Whiton's profile picture
Josh Whiton1 month ago

Love Pocket, but can you train the English model to say "live" with a hard "i" when appropriate? In my experience, it incorrectly says it with a soft 'i' 90% of the time.

Spletni zvočnik 🔊's profile picture
Spletni zvočnik 🔊1 month ago

Anyone interested in doing Slovenian?

Owen Song's profile picture
Owen Song1 month ago

would really love to see audio tags on a kyutai model one day!

Bastien Gares's profile picture
Bastien Gares1 month ago

Vous êtes des crack 🫡

sour kiwi's profile picture
sour kiwi1 month ago

crazy release! thank you

The Tech Fuse's profile picture
The Tech Fuse1 month ago

Training TTS on one GPU and running it on CPU is a killer open stack.

Nathan Roll's profile picture
Nathan Roll1 month ago

you guys are cracked-- keep it up!

Shahar Nechmad's profile picture
Shahar Nechmad1 month ago

Wow. Under $200 to train a useful TTS model from scratch is way more accessible than I expected. How much clean speech data do you need before it starts sounding natural in a new language, not just reading it correctly?

Mena Botrous's profile picture
Mena Botrous1 month ago

i like the open approach here. training recipes plus cpu inference makes this a practical tts project.

cetusian's profile picture
cetusian1 month ago

does the 50k step number hold for a language with way less data? romanian is my usual wall

Habtamu Asefa's profile picture
Habtamu Asefa1 month ago

You made my day 😁

Ryan Edkins's profile picture
Ryan Edkins1 month ago

Running inference on any device's CPU is the part I care about, that's the difference between a demo and something you'd actually ship inside a product.

rehan's profile picture
rehan1 month ago

love it!!!

Bart S.'s profile picture
Bart S.1 month ago

That's cool 🔥

RG's profile picture
RG1 month ago

thats what i was waiting for so long

Vishal Singh 🥑's profile picture
Vishal Singh 🥑1 month ago

Amazing, thanks 🔥🙌

Ekrem's profile picture
Ekrem1 month ago

There goes my $300

Adrian Scott | A.I. + Business Upscaling's profile picture
Adrian Scott | A.I. + Business Upscaling1 month ago

exciting, thanks!

semonic's profile picture
semonic1 month ago

Thank you You made my day

Jikku Jose's profile picture
Jikku Jose1 month ago

So are you saying any language TTS can be built with this stack? Isn’t the tokenizer dependent on language?

Rudra Satani's profile picture
Rudra Satani1 month ago

The step-count breakdown is the useful part: WER under 1% at ~50k but not sounding synthetic until ~200k means intelligibility and prosody are separate curves, and WER alone says you're done long before you are.

bright's profile picture
bright1 month ago

thank you 🙏🏼

Prosper's profile picture
Prosper1 month ago

Finally the wait is over!

Sebastian Buzdugan's profile picture
Sebastian Buzdugan1 month ago

cpu latency is mostly memory bound, so model size decides whether this ships

Zaky Vids's profile picture
Zaky Vids1 month ago

@kyutai_labs I am not very tech savy and I am trying to learn python but it will be quite sometime before I can do these things is there any way for you guys to release a version or some way where we can train our own voice in the browser like via browser inference

Ayush's profile picture
Ayush1 month ago

thanks for this 🤑

영웅 (woogie)'s profile picture
영웅 (woogie)1 month ago

< $200 for a full TTS train is the part I keep thinking about. that's the 200k-step run on rented H100s?

Ivan Fioravanti ᯅ's profile picture
Ivan Fioravanti ᯅ1 month ago

Amazing 🤩 thanks thanks thanks!

刘朝 Zhao Liu's profile picture
刘朝 Zhao Liu1 month ago

Open-sourcing recipes and evals turns TTS from an API dependency into a reproducible lab experiment. The next frontier is voice quality, latency, and footprint together; WER under 1% is one axis when an agent must speak on CPU in real time.

Michael's profile picture
Michael1 month ago

Hi guys, I decided to train this model on my MacBook, and the README invited a bug‑fix PR because it wasn't supported out of the box, so I opened one after making it possible to train on my MacBook, but the maintainer closed it, saying that was an oversight since they can't support platforms they have no hardware to test. I hold no grudge; his comment ended warmly, and I just want to share it here in case anyone wants to use it to train on a Mac as well.

rasmus's profile picture
rasmus1 month ago

i might try!

HopeCCCCCCCCCCCCCCCC's profile picture
HopeCCCCCCCCCCCCCCCC1 month ago

under $200 for a full run is way lower than i would have guessed

Related Videos

What if your voice AI could interrupt you the moment it figured out your question - sometimes even before you finished asking it? Last week, I sat down with Neil, CEO of Gradium and co-founder of Kyutai , to talk about the future of speech-to-speech models and why he believes today's cascaded voice systems will soon look "archaic and brittle." Some highlights from our conversation: 🎯 How Kyutai built Moshi—a full duplex conversational AI with "negative latency"—in 6 months with just 4-6 people (while big tech teams had 10-20x the resources) 🧠 Why speech-to-speech models lose intelligence compared to their text counterparts (and what's being done about it) 📱 Pocket TTS: The first voice cloning model that runs on your phone's CPU—not GPU, CPU 🤖 Why robotics and spatial audio represent the next frontier (hint: current voice systems completely break in these environments) 👶 The efficiency gap: Babies learn to speak fluently from <5,000 hours of audio. Current models train on millions of hours. We're doing something wrong. My favorite vision from Neil? The first truly contrarian AI that interrupts you mid-sentence to tell you why you're wrong. Not just more natural conversation—but actually useful for testing ideas and playing devil's advocate. Full episode and detailed blog post linked in the comments 👇 What's your take - will speech-to-speech replace cascaded systems, or will modularity keep cascaded architectures dominant even as naturalness improves?

Brooke Hopkins

13,000 views • 7 months ago

GPT 5.6 SOL IS HERE! How to run your personal + business life with GPT 5.6 Sol + Codex (full 49 min masterclass) We tested it for 30 days and the video it's the CLEAREST look at the FUTURE of work: Here's what's possible once you set it up: 1. Your inbox becomes cards every morning, each with a summary and a reply drafted in your own voice. Y 2. Your Slack, meeting notes, and company updates can turn into one daily feed with a clear next action. It learns what you care about over time and rewrites its own prompts to get sharper. 3. You can give your agent its own email address, so your other tools and even your team's Slack bot email it directly and it just handles things. 4. You can have it watch you do a task once and turn it into a skill it repeats forever. 5. You can set a long goal and walk away. You can have it run for 20 hours straight, and fine-tune your own models, something that was out of reach for non-engineers 12 months ago. How to start: Open Codex, give it access to your computer, and ask it to suggest things it could do for you based on how you already work. Full episode on The Startup Ideas Podcast (SIP) 🧃 (thanks Dan Shipper for sharing your entire workflow and review of GPT 5.6) Start with one boring task, get it working, and build from there. You'll learn exactly how to make something similar. GPT 5.6 Sol is impressive. Sol (according to openAI benchmarks) is the best coding model out right now. It set a new state of the art on Terminal-Bench 2.1 at 88.8%, and its "ultra mode" hits 91.9%, beating Claude Opus 4.8, Fable 5, and even Mythos 5 this masterclass is 100% free, like always. For more The Startup Ideas Podcast (SIP) 🧃 Watch

GREG ISENBERG

156,806 views • 3 months ago

I just inscribed the FILTER module to Termina! You can now SEARCH and FILTER bitmap content and fields, powered by CRAWL and the WORLDTREE data! Go to: Try > filter content bitoshi blockamoto > filter type audio > filter address skynet > filter (type image) (timestamp 2023) > filter help Make sure you have the latest snapshot from WORLDTREE, or have run your own CRAWL and WORLDTREE (warning - these take a long time! I'll be looking into ways to make running these more accessible). Bring your own data: The help menu is your friend here! Please try as much as you can. You can filter single values, or multiple things at the same time with parenthesis. I've found so many inscriptions on Bitmap this way! Please have fun with it, and if you make a new inscription on Bitmap, be sure to run "crawl bitmap [range]" and "worldtree bitmap full [range]" on your district so you can search & filter, and make sure to add it to github via a pull request. This way we can share the latest snapshot data. If you want to build your own dataset, try using CRAWL and WORLDTREE. I warn you though, the CRAWL module especially will take days if you don't have 18 windows open each scanning a range (in which case it will take about 8 hours). The WORLDTREE module takes about 8 hours with just one window open, so probably can accellerate the speed with multiple windows. This is why it's easier to piggyback off the github snapshots, but in the future these scans will be baked into the fabric of the operating system, automatically updated by users exploration and constantly scanning in the background (if turned on). Maybe there's some synergy to be had in this regard with ZmAKin_1981.🟧 extension to make sure we always have latest data. Anyway, that's all for now folks! Please share your findings.

bitoshi blockamoto 🧱 BITMAP 🟧

16,472 views • 1 year ago