Skip navigation
We release a technical blog on training PocketTTS, our 100M parameter on-device TTS, through…
0:00 / 0:56

We release a technical blog on training PocketTTS, our 100M parameter on-device TTS, through…

kyutai

@kyutai_labs28K followers

Visit on X
537 likesDownloadOriginal post

24,596 views

We release a technical blog on training PocketTTS, our 100M parameter on-device TTS, through drifting, a recent one-step generative objective from Deng et al. Less than 1% WER, high quality and voice cloning. To the best of our knowledge, it's the first speech model, and the first autoregressive model, trained this way.

11 Comments

Comments from the original post on X (Twitter)

  • kyutai's profile picture

    @kyutai_labs

    PocketTTS speaks in continuous latents, one every 80 ms, and a small head turns noise into the next latent in a single forward pass. We used to train with LSD, another one-step flow matching loss, but this required Jacobian-vector products at training time. This is not needed anymore with the drifting recipe.

    11 likes

  • kyutai's profile picture

    @kyutai_labs

    The key ingredient we came up with to make it work is a learned kernel temperature. The full write-up with the recipe and all the ablations is here : Code: Original Drifting paper:

    21 likes

  • Aïna's profile picture

    @ainafahy

    Can I just say this little guy is so cute

    4 likes

  • LastResort's profile picture

    @LastResort48

    @grok which language it supports?

    1 like

  • {stc}'s profile picture

    @stcisgood

    nice work playas

  • Francisco Carlos Erra's profile picture

    @francarloerra

    🙂

  • Eman Khoubian's profile picture

    @emankhoubian

    less than 1% WER on TTS means a speech recognizer heard the words. it says nothing about whether the cloned voice still sounds right at minute 20, which is where synthetic voices break.

  • Yazid Janati's profile picture

    @yjelid

    @YentiCollin @samsongvch

  • Vi VL's profile picture

    @ViVL787033

    Есть поддержка realtime?

  • mdzor's profile picture

    @mdzor

    @LocallyAIApp 🙈

  • semonic's profile picture

    @Semonic_

    Awesome i love this project I just wish you guys jad used more and better training data